Sharing compile-time constant values across templates

[jeskew]

Issue summary

When users want to share or standardize values across multiple templates, one pattern that many turn to is the "shared constants module." Similar to how applications will often define symbols representing invariant values in a single place in code (such as Bicep.Core.LanguageConstants in the Bicep compiler), a template author may define some known invariants as outputs in a dedicated module:

constants.bicep

output freezingPointC int = 0
output freezingPointF int = 32
output boilingPointC int = 100
output boilingPointF int = 212

This is extremely convenient for users. Since modules support parameterization, this pattern can also be used to follow standardized naming conventions:

names.bicep

param env 'dev' | 'test' | 'prod'

output rgName = 'contoso-app-${env}-group'

From ARM's perspective, however, using modules to share constants is undesirable. Modules become nested deployments, which impose a runtime cost in terms of orchestration and persistence. As we look into how to share additional values across templates (including user-defined types and functions), Bicep should offer a way to share values at compile time rather than relying on ARM to share values at run time.

Proposed remediation

Bicep should add two new keywords for working with sharable symbols: import and export. To avoid confusion with Bicep extensibility control statements, the provider registration statement should be changed from import 'kubernetes@1.0.0' to provider 'kubernetes@1.0.0'.

What makes a value sharable?

In order to be sharable, a value MUST:

• Be declared as a named symbol.
• Be a compile-time constant.
• NOT be observable from the deployment artifact itself.

The last stipulation is added because an import statement should not silently add to a template's effects or to its public contract. This precludes the sharing of module and resource symbols (which add a side effect (a deployment) to a template), as well as of param and output symbols (which add to the input and output data of a template). Outputs, modules, and resources can be "shared" via resource references. Some variables will be sharable, as will all user-defined types. Though still under development, user-defined functions are expected to be sharable as well. (This proposal assumes that functions will be declared with the func keyword, though that is not final).

Keyword	Sharable?	Rationale
param	never	Unclear meaning. Should the consuming template declare a parameter with the same name? Also, parameters almost never have compile-time constant values.
output	never	Unclear meaning. Should the consuming template declare an output with the same name?
module	never	A module is an action (a deployment), not a value.
resource	never	A resource is an action (a deployment), not a value. Even existing resources have a representation in the ARM deployment graph.
var	sometimes	If the value can be folded to a constant at compile time, sharing is allowed.
type	always	Types are always compile-time constants.
func	always	User-defined functions can't refer to parameters, variables, or other user-defined functions, nor may they use resource references. As such, they should always be compile-time constants. (Subject to revision as #9239 progresses.)

import

The import keyword will be used in a template that wishes to consume a shared value.

Example usage

Assuming a file mod.bicep with the following content:

export var foo = 'foo'
export var bar = 'bar'
export var baz = 'baz'

// template continues and includes parameters, resources, and outputs

To import the foo, bar, and baz symbols, a template would include an import statement:

import { foo, bar, baz } from './mod.bicep'

var bazAlias = baz

This would cause the compiled template to include the definitions of foo, bar, and baz as if they had been originally defined within the consuming Bicep template. The compiled template might look like the following:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "variables": {
    "foo": "foo",
    "bar": "bar",
    "baz": "baz",
    "bazAlias": "[variables('baz')]"
  },
  "resources": []
}

The above is the same template that would be compiled from the following Bicep:

var foo = 'foo'
var bar = 'bar'
var baz = 'baz'
var bazAlias = baz

There would be no runtime representation of the mod.bicep that had been imported; only of the included symbols.

Aliasing imported symbols

import will support aliasing via the as keyword:

import {
  foo as fizz,
  bar as buzz,
  baz as pop,
} from './mod.bicep'

var buzzAlias = buzz

import could also support importing all includable symbols under a provided namespace using syntax borrowed from recent versions of ECMAScript:

import * as mod from './mod.bicep'

var foo = mod.foo

export

The export keyword will be used in templates that produce shared values. export must precede a statement that declares named symbol with a compile-time constant value.

Examples

The following examples would all be permitted:

// The assigned value of bar is a constant, so no problem here
export var bar = 'foo'

// A symbolic reference is used, but the compiler can fold it down to a constant ('foo'), so no need to block
export var foo = bar

// Any variable whose type is a constant, literal value can be exported
export var interpolated = 'abc${foo}123'

// Types are always exportable
export type dict = {
  *: string
}

type myPrivateType = string
// Permitted even though an unexported symbol is within the type closure.
// 'myPrivateType' will be copied into the including template under a generated name and will not create a symbol that can be used.
export type myList = myPrivateType[]

The following usages would all be prohibited:

// disallowed even though this symbol has a compile-time constant value
export param foo 'foo' = 'foo'

// disallowed because the variable is derived from a deploy time value
param anInput string
export var derived = 'foo${anInput}bar'

// disallowed because a resource symbol has a side effect
export resource yetAnotherVnet 'Microsoft.Network/virtualNetworks@2022-05-01' = {
  ...
}

// existing resources are also disallowed
export resource vnet 'Microsoft.Network/virtualNetworks@2022-05-01' existing = {
  name: 'vnet'
}

// disallowed because module symbols also have a side effect
export module mod './mod.bicep' = {
  ...
}

// disallowed even though this symbol has a compile-time constant value
export output myOutput string = 'constant'

Should values be includable by default?

An alternative to the export keyword would be to make all symbols in a template consumable by default. I believe we should avoid doing so for two reasons:

1. Making values visible by default makes all a template's types and variables part of its public contract. Removing or changing the value of a var symbol would become a backwards incompatible change.
2. For symbol types that are sometimes exportable, reporting an error on the export keyword when the symbol is not a compile-time constant is a better user experience than reporting an error wherever the symbol is imported.

Alternative keywords to consider

import and export will likely be familiar to many Bicep users because it is used in the same way in JavaScript modules, but users familiar with JS might not realize that import will actually be copying the statements declaring the imported symbols into their template. (The mechanism used will be a little bit more sophisticated than simple copy/paste, but under the hood, import in Bicep will function more like include/require in PHP or #include in C than like import in JS or Python) Alternative keywords to consider:

• include/public
  • Better reflects the mechanism of copying symbol declarations into the include-ing template, but doesn't have the panache of import/export.
  • Also mixes parts of speech (include is a verb and public is an adjective) instead of using two verbs; this is reflective of the underlying effect of the statements in the compiled JSON template, so not necessarily a bad thing.
• use/share
  • Pithy
  • Parts of speech are matched, but it's perhaps not as intuitive that use/share are a pair (unlike import/export, which are pretty obviously two sides of the same coin)

Even if we opt to use use/share, include/public, or another keyword pair, we should still change the keyword used for provider registration from import to provider. Having both import and include as keywords but having them do very different things is likely to be a source of confusion, especially for new users.

Alternative approaches

Rather than introducing two new keywords, it may be possible to repurpose an existing statement type (though doing so may still require introducing at least two new keywords).

Decide at compile time whether an import is an extensibility control statement or a macro

One hurdle this proposal must address is that Bicep already has an import statement, though it is used for registering extensibility providers that can be called during a deployment.

How are extensibility provider registration and compile-time constant sharing the same?

The Bicep team frequently uses the word 'types' to describe how import modifies the compilation environment (i.e., import allows the template author to deploy new kinds of resources), and a major goal of the current proposal is to support the sharing of 'types.' From a template author's perspective, it would be confusing to say that some 'types' and functions must be imported while others would require a provider statement.

How are extensibility provider registration and compile-time constant sharing different?

When we talk about the 'types' that are introduced via an extensibility provider registration, we are refering to the varieties of resources that can be deployed in a template. Crucially, extensibility provider registration creates no new symbols for such 'types' but does allow more flexibility in the creation of resource symbols. For example, the endpoint symbol in the following template can only be created because the kubernetes provider has been registered in the first line:

import 'kubernetes@1.0.0'

resource endpoint 'core/Endpoints@v1' = {
  metadata: {
    ...
  }
}

'core/Endpoints@v1' is not a symbol and cannot be used as a parameter or output type, but we often call it a 'type' because it would be used to populate the type and apiVersion properties of the resource in the compiled JSON template.

Compile-time constant sharing, however, is meant to be executed at compile time (like a C macro) and leave no reference to the target of the import statement in the compiled artifact. The 'types' that would be pulled in are used exclusively for input and output validation but introduce no new deployment capabilities. The ARM runtime uses user-defined types for deployment pre- and post-condition verification, whereas ARM has no knowledge whatsoever of resource types. This distinction is muddied somewhat in Bicep, as the compiler will provide advisory validation of resource body data and resource property access, but in the ARM engine, such validation is left entirely up to the resource (or extensibility) provider.

What would using the same syntax for both extensibility provider registration and compile-time constant sharing look like?

The existing import statement would be unchanged. Currently, the string identifying what is imported must refer to well-known, named provider or to an OCI artifact containing a provider manifest:

import 'kubernetes@1.0.0'
import 'br:example.acr.io/privateProvider:1.0.1'
import 'br/example:anotherPrivateProvider:1.0.2'

These statements create three namespace symbols (kubernetes, privateProvider, and anotherPrivateProvider, respectively), each of which may be renamed using the as keyword:

import 'kubernetes@1.0.0' as k8s
import 'br:example.acr.io/privateProvider:1.0.1' as myProvider
import 'br/example:anotherPrivateProvider:1.0.2' as otherProvider

The Bicep compiler could treat an import statement pointing at an ARM or Bicep template as a different kind of statement, one that could pull in user-defined types, exported variables, or user-defined functions. The targeted template could exist either in a local file or in a Bicep module registry:

import './foo.bicep'
import 'br:example.acr.io/aModule:1.0.1'
import 'br/example:anotherModule:1.0.2'

Each of these statements would create a local namespace symbol (foo, aModule, and anotherModule, respectively) containing exported types, variables, and functions as top-level members of the namespace. The symbol associated with the namespace could be chosen with the as keyword, and the with clause (used to provide configuration to extensibility providers) would not be permitted.

import './foo.bicep' as bar

param customTypedParam bar.myCustomType

output anOutput bar.anotherCustomType

Only symbols introduced with the export keyword would be added to the imported namespace, and such symbols could be dereferenced either with or without the namespace, with the namespacing prefix of bar. only being required for disambiguation.

Disadvantages

Syntactic ambiguity

Repurposing the import keyword in this way would make static analysis and compilation more difficult, as the effect of an import statement would be entirely dependent on the content of what was imported. The strong distinction between importing an extensibility provider and importing constant values from another template would still exist in ARM JSON templates: the template emitter would, depending on whether the imported target was an extensibility provider or another template, either add to the imports block of the compiled template (in the case of a provider import) or copy the targets exported values into the compiled template (in the case of a template import).

There is unlikely to be any artifact that is ambiguously either an extensibility provider or a template, although this is possible with an OCI artifact. To disambiguate, we can require an extra keyword for constant value imports, e.g. import static 'foo.bicep' (to follow Java's example) or import constants 'foo.bicep' (taking inspiration from TypeScript's type-only imports). Indeed, we could require the static secondary keyword for every usage of import meant to cause compile-time constant sharing, which would allow the Bicep grammar to more precisely specify when the with clause of an import statement is and is not allowed. This would also alleviate any concerns about introducing syntax whose representation in the intermediate language is indeterminate, since import and import static would effectively be different keywords.

Semantic ambiguity

Assume a file mod.bicep that contains some exported constants alongside resource statements. import 'mod.bicep' will not execute the resource deployments described in mod.bicep, but will this be clear to users? The behavior difference between import 'mod.bicep' and module mod 'mod.bicep' = ... will need to be made clear via documentation, and I expect this distinction to be somewhat confusing to newcomers to Bicep. We could introduce a new file extension & artifact type that only allows type, func, and var statements and only permit those artifacts (and not standard Bicep templates) to be imported, but this feels like a lot of ceremony to impose in order to avoid a small amount of ambiguity. (If we do adopt this approach, we should strongly consider abandoning Bicep type syntax in favor of an externally maintained standard like JSON schema; there's no point in forcing users to learn a single-purpose syntax if it doesn't need to integrate with existing Bicep templates.)

The added ambiguity when comparing import 'mod.bicep' to import 'kubernetes@1.0.0' muddies the water further, since these two statements would have the same syntax but wildly different semantics, with the only distinguishing characteristic being the import statement's target. Experienced Bicep users would learn to distinguish the two statement types because the targets look different at a glance, though this is of course not the case for import statements targeting an OCI artifact ID. We may of course decide that the semantic difference between import 'mod.bicep' and import 'kubernetes@1.0.0' is an implementation detail only of concern to the ARM runtime. However, we will likely need to introduce some way for "power users" to disambiguate if they are concerned about template size limits and/or the runtime effects of extensibility provider registration (i.e., via an import static-like statement), especially if we expect OCI artifacts that are ambiguously either Bicep modules or extensibility providers to proliferate.

Less control over tree-shaking and aliasing

The import statement proposed in the first part of this document requires the template author to pick which symbols will be imported from the target and provides an opportunity to alias each one individually. The extensibility provider registration syntax, however, creates a single symbol for the imported namespace. Bicep only requires users to "fully qualify" a symbol (specify its namespace via the <namespace>.<symbol> syntax) if there is a locally declared symbol of the same name or if multiple imported namespaces declare a symbol of the same name. The import statement lets template authors alias the namespace symbol, but not any symbols contained in the namespace. This may lead to more verbose templates in some cases.

The proposed import {foo} from 'bar.bicep' syntax furthermore allows the compiler to aggressively tree-shake the symbol table of bar.bicep so that only the declarations necessary for foo to be a valid symbol are copied into the host template. A naive implementation of the alternate import 'bar.bicep' syntax would copy all exportable symbols from bar.bicep into the host template, although the compiler could be enhanced to perform tree-shaking based on which symbols from bar.bicep are dereferenced in the host template. This could potentially be seen as an advantage of the import 'bar.bicep' syntax, as import {foo} from 'bar.bicep' forces the template author to identify which symbols will be used in the host template, even though that's something the compiler can trivially determine given the import 'bar.bicep' syntax.

Share compile-time constants via const module statements

Bicep already has a mechanism for referencing another template: as a module that will be executed as a nested deployment.

How are module and import the same?

Both target another template and allow the parent template (the one in which the module or import statement occurs) to dereference symbols of the targeted template (outputs in the case of module; constants in the case of import). As noted above, it is not uncommon to use modules today to share values across templates.

How are module and import different?

A module statement creates a Microsoft.Resources/deployments resource that will be deployed. Like all other resources, a module must have a name property, which is used to create an identifier associated with the resource, allowing the resource to be dereferenced by ID rather than by symbol. This has a runtime cost in the ARM engine. The expectation with import is that it will have no runtime cost and will be resolved entirely at compile time. import also has no declaration body, as it has no properties.

What would importing constant values via module statements look like?

Because module statements already target templates, users would need to disambiguate between nested deployment modules and constant-value-import modules. This could be accomplished with an extra keyword preceding the module keyword, e.g.,

const module mod './mod.bicep'

Any alternative to const (such as inline, static, constonly, constants, etc.) could be used instead, so long as it does not conflict with any existing Bicep keyword.

I'll refer to the existing module statement as a "deployed module" and the strawman syntax above as a "const module". A "const module" statement would introduce a new namespace symbol (mod in this case), and exported symbols in the const module could be dereferenced either with or without the namespace, with the namespacing prefix of mod. only being required for disambiguation:

const module mod './mod.bicep'

type widget = {
  ...
}

param myParam customType // "customType" is defined in mod.bicep
param another mod.widget // "mod.widget" must be fully qualified to disambiguate from the locally-declared widget symbol

One advantage of this approach is that it would make sense to expose exported values from "deployed modules," too, since I expect many users would want to access a deployed module's exported types in addition to its outputs. One common case might be if a template author wants to accept a parameter composed of the parameter types of a module:

// mod.bicep

export type storagePropertiesType = {
  ...
}

param storageProperties storagePropertiesType

export type lbPropertiesType = {
  ...
}

param lbProperties lbPropertiesType

export type fooOrBarType = 'foo' | 'bar'

param fooOrBar fooOrBarType

// main.bicep

param modProperties {
  storageProps: mod.storagePropertiesType
  lbProps: mod.lbPropertiesType
}

module mod 'mod.bicep' = {
  name: 'mod'
  params: {
    storageProperties: modProperties.storageProps
    lbPropertiesType: modProperties.lbProps
    fooOrBar: 'bar'
  }
}

With the import syntax proposed in the first part of this document, users would need to have both an import and a module statement to achieve this:

//main.bicep

import {storagePropertiesType, lbPropertiesType} from 'mod.bicep'

param modProperties {
  storageProps: storagePropertiesType
  lbProps: lbPropertiesType
}

module mod 'mod.bicep' = {
  name: 'mod'
  params: {
    storageProperties: modProperties.storageProps
    lbPropertiesType: modProperties.lbProps
    fooOrBar: 'bar'
  }
}

(Bicep could opt to support pulling in types via either an import or a module statement, with the distinction being that the latter also deploys the module. It would probably be worth gathering user feedback on whether users actually want this feature and on whether supporting constant sharing via two separate mechanisms, each with its own side effects, is confusing.)

Disadvantages

Could this overcomplicate the module statement?

There has also been some discussion of adding an inline prefix keyword to module statements as a way to share code without creating nested deployment resources. If both inline and const keywords were added, they would be incompatible, and the module statement overall would become a fairly complex beast.

Less control over tree-shaking and aliasing

The import statement proposed in the first part of this document requires the template author to pick which symbols will be imported from the target and provides an opportunity to alias each one individually. The "const module" syntax, however, creates a single symbol for the imported module. This change will require Bicep to treat module symbols as a form of namespace. Bicep only requires users to "fully qualify" a symbol (specify its namespace via the <namespace>.<symbol> syntax) if there is a locally declared symbol of the same name or if multiple imported namespaces declare a symbol of the same name. The const module statement lets template authors alias the module symbol, but not any symbols contained in the namespace. This may lead to more verbose templates in some cases.

The proposed import {foo} from 'bar.bicep' syntax furthermore allows the compiler to aggressively tree-shake the symbol table of bar.bicep so that only the declarations necessary for foo to be a valid symbol are copied into the host template. A naive implementation of the alternate const module mod 'bar.bicep' syntax would copy all exportable symbols from bar.bicep into the host template, although the compiler could be enhanced to perform tree-shaking based on which symbols from bar.bicep are dereferenced in the host template. This could potentially be seen as an advantage of the "const module" syntax, as import {foo} from 'bar.bicep' forces the template author to identify which symbols will be used in the host template, even though that's something the compiler can trivially determine given the "const module" syntax.

[jeskew]

Some notes from today's team discussion:

• Should we use import and export for symmetry?
  • But what about importing extensibility providers?
  • If we don't combine the import statement to support both extensibility providers AND compile-time symbol sharing, we should consider picking a different keyword for extensibility
  • include and pub isn't exactly symmetrical, but that combo is an option.
• This feature alongside extensibility creates some confusion.
  • Users import an extensibility provider, giving the template the ability to deploy specific kinds of resources. Users might then also include a local file. which copies over some variables and/or types at compile time.
  • Confusingly, both import and include can pull in "types"
    • import will add type information used by Bicep to validate resource bodies.
      • Resource body validation will only produce warnings because we can't be sure the types that will be enforced by the provider when the template is deployed are accurately described in the provider's manifest
    • include can add type alias symbols that can be used in param and output statements.
      • Parameter and output types are enforced by ARM at runtime, and violating a user-defined type raises a compiler error.
        • The type is defined in the template itself, so we're not worried about drift between compile time and deploy time in the same way we are for resource types.
    • Are we drawing a false distinction between resource types and user-defined types?
      • If both are types, why do we treat them differently in the compiler?
    • Conversely, would this confusion get cleared up if we were better at naming things?
      • User-defined types and resource types present themselves very differently in a template, so why do we call them both "types"?
      • A template author can write a user-defined type, but they will likely never write a provider manifest.
• The original proposal allowed sharing existing resources. This should be disallowed because existing resources have a representation in the deployment graph.

[rouke-broersma]

I'm noticing that Bicep is more-and-more starting to look like an actual programming language, I'm wondering if at some point it shouldn't be reconsidered to simply co-opt a subset of an existing programming language syntax instead of mix-and-matching from different languages or coming up with Bicep specific terms. It would improve clarity imo as Bicep is becoming more and more complex as a language. This will start to cause confusion during development, especially if you're 'full stack'.

This would also remove some of the need for discussion. For example if it was decided to more or less follow javascript syntax and keywords import might have never been chosen for provider extensibility and would be available for import/export.

[anthony-c-martin]

Some additional thoughts:

• Looks like you captured this one already, but being able to export existing resources feels like we'd have to introduce significant limitations to make it work - in practice, you would need to capture scope & name - and in my experience these often tend to not be compile-time constants. I don't see a very solid use case for this, and worry about the confusion between when you can/can't use this feature - so feel like it's worth omitting for now.
• It might be useful to annotate a var with a type, such that a module is able to provide functionality akin to an interface - to avoid tight coupling between the calling & callee module (e.g. so the callee module is able to modify what data it returns over time, without affecting the type that is exposed to the calling module). There may be cases where the module author doesn't want the consumer to know/care about the implementation details, and wouldn't want to break the consumer by revving the version.
• It would be good to mock up how this would work with the functions proposal, just to try and ensure the two features are compatible. Functions feels like an important use case for code reusability.
• The inlining capability may need to be a bit more sophisticated than copy+paste - it may need to take scoping into consideration - e.g.

  var foo = 'abc'
  // will 'foo' be represented as a variable in the target file, or will this be evaluated?
  export var bar = { foo: foo }

  I can see the argument here for just fully evaluating variables, but if we want to allow scoping for functions, we can't rely on evaluating expressions at compile time:

  var baseName = 'blah'
  func generateAName(suffix: string) => `${baseName}-${suffix}`

  There's also the possibility of blowing up the generated output if we do full evaluation:

  export var foo = map(range(0, 10000), 'loooooongstring${i}')

• If trying to avoid import for confusion with extensibility, I like include/public. I'm not concerned about the lack of symmetry with private. If trying to be even more differentiated, then other options could be use/public (although, confusion with params files), or require/public.

[jeskew]

It would be good to mock up how this would work with the functions proposal, just to try and ensure the two features are compatible. Functions feels like an important use case for code reusability.

I left that section a little hazy because UDFs in Bicep are still a work in progress, and there are some decisions that haven't been made that would have a big impact on how include would work with UDFs (specifically, can UDFs refer to variables and parameters?).

I think an iterative approach for this proposal would be best, since there are at least three areas that need further discussion:

• Which keywords should we use?
• How does constant folding for public vars work?
• How do we move UDF closures between templates?

If this proposal were implemented for types, then for vars, then for UDFs, that would allow those questions to be addressed separately.

The inlining capability may need to be a bit more sophisticated than copy+paste - it may need to take scoping into consideration - e.g.

var foo = 'abc'
 // will 'foo' be represented as a variable in the target file, or will this be evaluated?
export var bar = { foo: foo }

My initial thought (based on no prototyping, so subject to change!) was that export/public would only be permitted on variables whose type is a literal per TypeHelper.IsLiteralType, and that we would convert the literal value to Bicep syntax (similar to the TypeSymbol => JToken conversion we do to evaluate the return type of transformation functions).

So in that case, the foo variable wouldn't have any representation in a template that pulls in bar. include { bar } from '<path>' would result in the same compiled JSON as var bar = { foo: 'abc' }.

I can see the argument here for just fully evaluating variables, but if we want to allow scoping for functions, we can't rely on evaluating expressions at compile time:

var baseName = 'blah'
func generateAName(suffix: string) => `${baseName}-${suffix}`

I think the current draft of the UDF proposal doesn't allow variable or parameter references, but if that were to change, we would likely need to create a special kind of closure scope to keep functions sharable. The key goal with include is not to pollute the consuming template's namespace with any symbols that aren't explicitly pulled in. We can always add ARM variables whose names aren't valid BIcep identifiers to keep them separate (though that will raise some problems for decompilation).

There's also the possibility of blowing up the generated output if we do full evaluation:

export var foo = map(range(0, 10000), 'loooooongstring${i}')

That's true, but whether the expression or its evaluated type is longer is really dependent on the expression. 1 + 1 is five times as many characters as 2, and I would wager that most transformation functions are longer than the changes they effect on the evaluated type (e.g., compare to toLower(trim(' A ')) vs 'a' or take([1, <another 9,999 elements>], 1) vs 1. We decided not to have range return a tuple specifically because range(x, 10000) would be less useful as a 10,000 member tuple than as just as int[], and I think relying on the constant folding in the type system allows room for that kind of discretion. As this proposal progresses, though, we might want to carve out special handling for specific cases like range and decouple the proposal from the type system's constant folding.

If trying to avoid import for confusion with extensibility, I like include/public. I'm not concerned about the lack of symmetry with private. If trying to be even more differentiated, then other options could be use/public (although, confusion with params files), or require/public.

+1, include/public is unambiguous.

[dazinator]

Some more novel alternatives for fun:

share | expose | surface / use | consume | require | reference | link

I like the idea of share | use combo!
I thought perhaps you might want to reserve the word public just in case you ever introduce a concept similar in nature to access modifiers in c# in future.. (public | internal | private etc) - as access to objects and variables (of any type or value) is one thing, where as exporting a symbol for compile time inclusion is a different use case with a specific set of restrictions.

[miqm]

👍🏻 on share/use

[WhitWaldo]

Would the idea of being able to expose compile-time foldable constants include those values which are the output of loops?

[jeskew]

Would the idea of being able to expose compile-time foldable constants include those values which are the output of loops?

I think it should. There may be some work that needs to be done on loop type inference to make that possible.

[Jaykul]

You could borrow from shell languages and use export | source ;-)

[dciborow]

You could borrow from shell languages and use export | source ;-)

I don't have an opinion on the terms chosen, but my one nitpick is I think the "from" piece of the declartion should go before the "include". This is how the Python style for imports is done, with the optional 'from' keyword before the 'import'. When including many imports, there is not a clear way to sort the imports. If the imports are ordered by the first imported item, this could lead to the imports being shuffled around as more things are added by a user.

Python

Python offers a variety of ways to define imports. Generally, it's advised to avoid using "*", which leads to a decrease in performance, but it is still a valid option.

import os
import shutil
import pandas as pd
from subprocess import *
from pathlib import Path as PathLibPath
from tempfile import mkdtemp, rmdtemp as rmdir

Bicep

Using that as inspiration, here are some examples of how it would look in Bicep.

Import specific items directly.

from './mod.bicep' import foo, buzz, baz

Import specific items with alais.

from './mod.bicep' import {foo as fizz, bar as buzz, baz as pop} 

param fooParam fizz
param barParam buzz
param bazParam pop

Import all items

In python, from './mod.bicep' import * is the same as 'import './mod.bicep'. Python does not let 'as XXX' to be applied to the *, which I found a little confusing in the example.

import './mod.bicep' as mod

param input mod.foo

Comparing to proposal syntax

When including many imports, there is not a clear way to organize the imports. If the imports are ordered by the first item, this could lead to the imports being shuffled around as more things are added by a user.

import {foo as fizz, bar as buzz, baz as pop} from 'mod.bicep'
import {name as ComputeName, sku as ComputeSku, location as ComputeLocation} from 'typesCompute.bicep'
import {name as StorageName, sku as StorageSku, location as StorageLocation} from 'typesStorage.bicep'

param fooParam fizz
param barParam buzz
param bazParam pop

param computeName ComputeName
...

Starting with the from clause, we can order all the imports alphabetically by the file names. Now, even if we add more imports to typesStorage like 'aDefaultConfig', we do not have to reorder the statements.

from 'mod.bicep' import {foo as fizz, bar as buzz, baz as pop} 
from 'typesCompute.bicep' import {name as ComputeName, sku as ComputeSku, location as ComputeLocation}
from 'typesStorage.bicep' import {name as StorageName, sku as StorageSku, location as StorageLocation}

param fooParam fizz
param barParam buzz
param bazParam pop

param computeName ComputeName

Other langauges

This is pretty similar to other languages like Java and Scala. Though, they do not use the "from" keyword. I do think the 'from' is a bit easier to read, once you are used to it.

import users.*  // import everything from the users package
import users.given // import all given from the users package
import users.User  // import the class User
import users.{User, UserPreferences}  // Only imports selected members
import users.UserPreferences as UPrefs  // import and rename for convenience

[dciborow]

Will we be able to use registry-based modules in the from clause?

For example, I would like to be able to create a collection of my common types in our registry, and reuse them in my templates.

from 'br/public:types/storage-acount:0.0.1' import types as StorageTypes

param location StorageTypes.location
param name StorageTypes.name
param newOrExisting StorageTypes.newOrExisting
param isZoneRedundant StorageTypes.isZoneRedundant

module storageAccount 'br/public:storage/storage-acount:0.0.1' = {
  name: 'mystorageaccount'
  params: {
    location: location
    name: name
    newOrExisting: newOrExisting
    isZoneRedundant: isZoneRedundant
  }
}

output storageProperties StorageTypes.storageProperties = storageAccount.outputs.storageProperties

This could be hard, because I don't think the user-defined type definition would get pushed to the registry. Maybe, we could consider exporting the type. Maybe, we could consider adding the type definitions to the metadata of the ARM template.

If it is particularly difficult to get the type information from a registry module, at least being able use a URL with from would enable the essential functionality. I could leverage aka.ms urls to make it look pretty.

from 'https://raw.githubusercontent.com/Azure/bicep-registry-modules/main/modules/storage/storage-account/main.bicep' import types as StorageTypes

or

from 'aka.ms/bicep-storage-types' import types as StorageTypes

[dciborow]

One last thought, there could be cases where we do not want to make everything in a bicep file importable, and instead we may want to make something 'private'.

Here, I would also suggest looking at how this has been in done in Python. Basically, the language has no 'private' keyword. By notation, a method or variable name that is prefixed with "_" is considered private (ex: def _my_private_method(): return "Something"). Programmers are freely able to use private methods if they want to, but static analysis tools will add a warning.

I really like how this 'psuedo-private' is done in Python. Unlike languages like C#, or Java, which have very strict 'private' keywords, in Python it is super easy to test private methods. Having 'private' be part of the static analysis, instead of as part of the compilation rules, gives users the most flexibility.

In the bicepconfig, users can choose if using private items causes a error or just a warning.

[jeskew]

@dciborow I'll bring the placement of from up at a Bicep discussions meeting. (The initial proposal (including the placement of the from clause) is based on TypeScript/ES6 modules.) As for a private keyword, my personal preference would be to make sharing strictly opt-in (with a public or export keyword) rather than opt-out, mostly because type and var statements exist today and aren't sharable between templates.

@rouke-broersma I missed your comment earlier, but you may want to look at Farmer. It's an F# library that lets you write ARM templates as F# scripts. We're very committed to making a Turing-incomplete DSL that is approachable to an audience composed of both developers and systems administrators, but I acknowledge that no solution will be the best choice for every team. We try to keep an up-to-date listing of alternatives to Bicep that target the ARM engine and ecosystem here.

[jeskew]

Based on team discussions, I'm including two alternative proposals below, one that incorporates the semantic described above into import statements and one that incorporates it into module statements. Each is a separate comment so that it can get its own reactions.

Sticking the discussion label back on the issue so that we can talk these over at an upcoming team meeting.

[jeskew]

This content was moved to be the first alternative proposal in the issue description.

### Sharing compile-time constant values across templates via `import`

One hurdle this proposal must address is that Bicep already has an import statement, though it is used for registering extensibility providers that can be called during a deployment.

How are import and include the same?

The Bicep team frequently uses the word 'types' to describe how import modifies the compilation environment (i.e., import allows the template author to deploy new kinds of resources), and a major goal of the current proposal is to support the sharing of 'types.' Additionally, an import statement will create new function symbols in the template's namespace. From a template author's perspective, it would be confusing to say that some 'types' and functions must be imported while others would need to be included.

How are import and include different?

When we talk about the 'types' that are introduced via an import statement, we are refering to the varieties of resources that can be deployed in a template. Crucially, import creates no new symbols for such 'types' but does allow more flexibility in the creation of resource symbols. For example, the endpoint symbol in the following template can only be created because of the import statement registering the kubernetes provider:

import 'kubernetes@1.0.0'

resource endpoint 'core/Endpoints@v1' = {
  metadata: {
    ...
  }
}

Similarly, any function symbols brought in by an import statement must be executed during a deployment by the registered extensibility provider. import is a "meta-directive" to the ARM deployment engine to modify its own runtime so that additional, runtime-only behavior can be exercised during a deployment.

include, however, is meant to be executed at compile time (like a C macro) and leave no reference to the target of the include statement in the compiled artifact. The 'types' that would be pulled in via include are used exclusively for input and output validation but introduce no new deployment capabilities. The ARM runtime uses user-defined types for deployment pre- and post-condition verification, whereas ARM has no knowledge whatsoever of resource types. This distinction is muddied somewhat in Bicep, as the compiler will provide advisory validation of resource body data and resource property access, but in the ARM engine, such validation is left entirely up to the resource (or extensibility) provider.

In terms of function symbols, include will need to modify the template to contain the definition of any user-defined function that is pulled in, whereas functions exposed via an extensibility provider would not be defined in the template and would be executed by said provider during a deployment.

What would importing constant values look like?

The existing import statement would be unchanged. Currently, the string identifying what is imported must refer to well-known, named provider or to an OCI artifact containing a provider manifest:

import 'kubernetes@1.0.0'
import 'br:example.acr.io/privateProvider:1.0.1'
import 'br/example:anotherPrivateProvider:1.0.2'

These statements create three namespace symbols (kubernetes, privateProvider, and anotherPrivateProvider, respectively), each of which may be renamed using the as keyword:

import 'kubernetes@1.0.0' as k8s
import 'br:example.acr.io/privateProvider:1.0.1' as myProvider
import 'br/example:anotherPrivateProvider:1.0.2' as otherProvider

The Bicep compiler could treat an import statement pointing at an ARM or Bicep template as a different kind of statement, one that could pull in user-defined types, exported variables, or user-defined functions (as described for include above). The targeted template could exist either in a local file or in a Bicep module registry:

import './foo.bicep'
import 'br:example.acr.io/aModule:1.0.1'
import 'br/example:anotherModule:1.0.2'

Each of these statements would create a local namespace symbol (foo, aModule, and anotherModule, respectively) containing exported types, variables, and functions as top-level members of the namespace. The symbol associated with the namespace could be chosen with the as keyword, and the with clause (used to provide configuration to extensibility providers) would not be permitted (at least not in the first iteration).

import './foo.bicep' as bar

param customTypedParam bar.myCustomType

output anOutput bar.anotherCustomType

Only symbols introduced with the export keyword would be added to the imported namespace, and such symbols could be dereferenced either with or without the namespace, with the namespacing prefix of foo. only being required for disambiguation.

Disadvantages

Repurposing the import keyword in this way would make static analysis and compilation more difficult, as the effect of an import statement would be entirely dependent on the content of what was imported. The strong distinction between importing an extensibility provider and importing constant values from another template would still exist in ARM JSON templates: the template emitter would, depending on whether the imported target was an extensibility provider or another template, either add to the imports block of the compiled template (in the case of a provider import) or copy the targets exported values into the compiled template (in the case of a template import).

There is unlikely to be any artifact that is ambiguously either an extensibility provider or a template, although this is possible with an OCI artifact. To disambiguate, we can require an extra keyword for constant value imports, e.g. import static './foo.bicep' (to follow Java's example) or import constants './foo.bicep' (taking inspiration from TypeScript's type-only imports).

But even if ambiguity is not a concern, we should be wary of introducing syntax whose representation in the intermediate language is indeterminate.

[jeskew]

This content was moved to be the second alternative proposal in the issue description.

### Sharing compile-time constant values across templates via `module`

Bicep already has a mechanism for referencing another template: as a module that will be executed as a nested deployment.

How are module and include the same?

Both target another template and allow the parent template (the one in which the module or include statement occurs) to dereference symbols of the targeted template (outputs in the case of module; constants in the case of include). As noted above, it is not uncommon to use modules today to share values across templates.

How are module and include different?

A module statement creates a Microsoft.Resources/deployments resource that will be deployed. Like all other resources, a module must have a name property, which is used to create an identifier associated with the resource, allowing the resource to be dereferenced by ID rather than by symbol. This has a runtime cost in the ARM engine. The expectation with include is that it have no runtime cost and be resolved entirely at compile time. include also has no declaration body, as it has no properties.

What would importing constant values via module statements look like?

Because module statements already target templates, users would need to disambiguate between nested deployment modules and constant-value-import modules. This could be accomplished with an extra keyword preceding the module keyword, e.g.,

constants module mod './mod.bicep'

I'll refer to the existing module statement as a "deployed module" and the strawman syntax above as a "constants module". A "constants module" statement would introduce a new namespace symbol (mod in this case), and exported (or public) symbols in the constants module could be dereferenced either with or without the namespace, with the namespacing prefix of mod. only being required for disambiguation:

constants module mod './mod.bicep'

type widget = {
  ...
}

param myParam customType // "customType" is defined in mod.bicep
param another mod.widget // "mod.widget" must be fully qualified to disambiguate from the locally-declared widget symbol

One advantage of this approach is that it would make sense to expose public values from "deployed modules," too, since I expect many users would want to access a deployed module's exported types in addition to its outputs. One common case might be if a template author wants to accept a parameter composed of the parameter types of a module:

mod.bicep

export type storagePropertiesType = {
  ...
}

param storageProperties storagePropertiesType

export type lbPropertiesType = {
  ...
}

param lbProperties lbPropertiesType

export type fooOrBarType = 'foo' | 'bar'

param fooOrBar fooOrBarType

main.bicep

param modProperties {
  storageProps: mod.storagePropertiesType
  lbProps: mod.lbPropertiesType
}

module mod 'mod.bicep' = {
  name: 'mod'
  params: {
    storageProperties: modProperties.storageProps
    lbPropertiesType: modProperties.lbProps
    fooOrBar: 'bar'
  }
}

Disadvantages

Like with repurposing the import keyword, we should be wary of introducing syntax whose representation in the intermediate language is indeterminate.

There has also been some discussion of adding an inline prefix keyword to module statements as a way to share code without creating nested deployment resources. If both inline and constants keywords were added, they would be incompatible, and the module statement overall would become a fairly complex beast.

[WhitWaldo]

I'm imagining types as used in Bicep as being quite similar to interfaces in C# in that they identify the shape of the implementing object and can be passed around like contracts - one then develops more against the interfaces and cares a lot less about whatever it is that ultimately implements it.

To that end, my preference at a high level is that we use keywords over conventions where possible. Within a module, everything remains private to that module by default (as it is today) unless the symbol is prefixed with a public keyword or otherwise emitted as an output.

The import keyword is already used for two use-cases: importing a named provider and referencing container artifacts. I'd propose one of two routes here:

1. Make import also support importing types and constants alike from other files. The shape of the argument can trivially allow inference of what's being referenced (all artifacts start with br: and their modules are explicitly pathed from there, all local modules are referenced with relative path notation and everything else is assumed to be a named provider). Make import a universal inclusion keyword across the board to avoid muddying the waters and introducing special keywords that are simply doing imports in various ways. That ship has sailed when import was repurposed already - don't suddenly change course now.

import * from './foo.bicep' as bar //Everything that is exported from foo is accessible from the 'bar' namespace
import { customType } from './foo.bicep' //Only the 'customType' is available in the current module from foo
import { customType as bar } from './foo.bicep' //Only the 'customType' is available from foo and it's called 'bar' when used in the file
import 'br:foo/bar@1.2.2' as bar //Everything that's exported from the module at foo/bar is accessible from the `bar` namespace
import 'kubernetes@1.0.0' //Standard named provider import
import { customType as foo } from './foo.bicep' as bar //Invalid syntax as the type already has an alias and the whole of the file isn't being used here

2. Or, leave import as it is. Today, it means that you're accessing something that's not locally yours - either a remote containerized artifact or a named provider (perhaps a custom named provider down the road). Instead, introduce a keyword that is used for all local type symbol/constant values. I would propose using especially coupled with the ES6/Typescript syntax:

using * from './foo.bicep' as bar //Everything that is exported from foo is accessible from the 'bar' namespace
using { customType } from './foo.bicep' //Only the 'customType' is available in the current module from foo
using { customType as bar } from './foo.bicep' //Only the 'customType' is available from foo and it's called 'bar' when used in the file
using { myConstant as c } from './foo.bicep' //The referenced symbol is a constant, so allow it to be used exclusively as a constant in this file
using { customType as foo } from './foo.bicep' as bar //Invalid syntax as the type already has an alias and the whole of the file isn't being used here

The downside to the using keyword instead of retaining import for everything is that it leave ambiguous which keyword should be used (import or using) when importing a type from a module contained in an extermal containerized artifact - since it's a container, should I use import or since I'm referencing a type should I use using? In this case, you might either distinguish with local (using)/not (import) or just combine the two:

using import { customType as bar } from 'br:foo/bar@1.2.2' //Remote source, but using the type

I expect that in my own deployments, I'll have whole files (modules?) that aren't intended to be deployed, but are just entirely filled with shared types. Rather than having to import all or none of them, I'd rather not pollute Intellisense all the itme and instead have the option to either import all via a wildcard or import select types that can be optionally aliased.

Regarding the module discussion, today, Bicep largely treats every file as a separate module that simply needs to be referenced in another file to access the outputs. I would again propose that the types be largely considered as just a compile-time feature (like in TypeScript) where, like interfaces in C#, they provide a shape that can be developed against for strongly-typed support, but where at runtime, it doesn't matter what ultimately fills the shoes.

I propose that constants be treated as fixed value types, to be made available like any other types via using statements instead of having yet another keyword. I don't fully follow what the purpose is of constants module mod './mod.bicep' Using the from syntax above, why can the module not simply be the source on the right of the from keyword, especially if the type is just a dev/compile-time constraint and the constant just folded into wherever it's used at compile time? I don't quite follow why there needs to be a rich support for actually pulling data out of modules beyond the outputs (excluding the value of outputting whole resources which I understand to be outside the scope of this discussion).

[jeskew]

The import keyword is already used for two use-cases: importing a named provider and referencing container artifacts.

I don't think this is accurate. The import keyword supports a single use case -- importing an extensibility provider so that additional kinds of resources can be deployed -- and an extensibility provider can either be defined in a container artifact or be one of a handful of "well known" providers. (The list of well known providers is currently kubernetes and az.) Extensibility is still in preview, so I'm not sure if that will change before the feature is considered generally available.

Make import also support importing types and constants alike from other files. The shape of the argument can trivially allow inference of what's being referenced (all artifacts start with br: and their modules are explicitly pathed from there, all local modules are referenced with relative path notation and everything else is assumed to be a named provider).

The ambiguity that I'm concerned about is that any OCI artifact ID (like 'br:foo/bar@1.2.2') could be an extensibility provider, a module, or both, and the only way to tell what should be done is to download the artifact and inspect its layers. In the case where an OCI artifact has layers allowing it to be both an extensibility provider and a module, there is an unnecessary runtime cost to hooking up the provider for extensibility if the template author only intended to use the target artifact's types. A local path is definitely a module and kubernetes or az is definitely an extensibility provider, but an OCI reference is ambiguous.

Make import a universal inclusion keyword across the board to avoid muddying the waters and introducing special keywords that are simply doing imports in various ways. That ship has sailed when import was repurposed already - don't suddenly change course now.

Extensibility is still in preview, so we have some flexibility there. It's possible extensibility providers could use another keyword (like using, provider, register, etc.).

One bit of feedback we got from Anders Hejlsberg was that we should avoid cases where the same Bicep syntax generates different ARM templates based on the values supplied to said syntax. If we can't tell what import 'br:foo/bar@1.2.2' will compile to without looking at the contents of br:foo/bar@1.2.2, that seems like a pretty big rift between Bicep syntax and the ARM intermediate language.

I don't fully follow what the purpose is of constants module mod './mod.bicep'

The proposals are separate alternatives. We can introduce a new keyword (like include), or we can try to shoehorn this functionality into an existing keyword (in which case it would either be import or module, but not both).

As an example, assume you have the following types.bicep file:

@minLength(3)
@maxLength(24)
type shortName = string

Assuming you have a name parameter that you want to make sure matches the shortName type, here's what that would look like under some of the proposals:

include: 

include {shortName} from 'types.bicep'
param name shortName

import:

import 'types.bicep'
param name types.shortName

or

from 'types.bicep' import shortName
param name shortName

module:

constants module types 'types.bicep'
param name types.shortName

I think we could probably make the types. in the second and fourth examples optional in most cases.

The constants keyword was meant to fill the role that type does in the TypeScript statement, import type { foo } from './bar.js'. Somebdy who is better at naming things would pick a better keyword than constants if we went with that option.

[dazinator]

Just chiming in to say that due to the natural proclivity to refer to these as "shared" values (as demonstrated above on this thread)
And due to the specifics around this use case being different from "module" and "import", I just wanted to add my vote for using a new keyword: share feels right to me, with its partner use.

use foo from `shared.bicep`
use bar as mybar from `shared.bicep`

using keyword has overlap with c# concepts but use is short and sweet and nails the intent. Also share feels better to me than export - export to where? share nails the intent better imho.

I also wonder how much real value there is in supporting aliasing given variable renaming is pretty easy in vs code? Still I guess it's a nice in principle..

[WhitWaldo]

I also wonder how much real value there is in supporting aliasing given variable renaming is pretty easy in vs code? Still I guess it's a nice in principle..

The greatest value in aliasing for me would be for those modules that I didn't write and don't own myself but still want to consume despite there being a name conflict.

[asilverman]

First, I want to acknowledge the amount of effort that it takes to come up with the design and deliberations in finding a good solution for this problem, all my praise to the people leading the effort and all the people acting as contributors.

I would like to propose a potentially controversial different approach I will elaborate below.

As I reason through the big picture, it appears that there is a concept of Bicep symbol that has 4 special cases:

1. A resource type
2. A user-defined type
3. A Bicep variable (whose assigned value is final)
4. A module (which is effectively translated to a resource type of type 'Microsoft.Resources/deployment')

We introduced the import keyword initially for providers and I think that it organically propagates to importing all other instances of a Bicep symbol.

Allowing the definition of user-defined types in a Bicep module (i.e. a .bicep file) creates as a consequence the problem that we are trying to solve in this proposal. Instead, I propose that we disallow defining user-defined types and constants in .bicep files and instead introduce a new file type .biceptypes (or an aesthetically equivalent name) to specify these kinds of symbols.

Doing so poses the following advantages:

• Clear separation between Bicep modules (aka. deployments) and other the concept of user-defined types and constants
• The import keyword can be used to include constants and types from a registry using a gesture similar to importing providers
• The module keyword remains unchanged semantically and in its behavior

Assuming this is accepted and adopted, an example .bicep file would look as following:

myTypes.biceptypes

type storagePropertiesType = {
  ...
}

type lbPropertiesType = {
  ...
}

type fooOrBarType = 'foo' | 'bar'

mod.bicep

import 'br/public/types/myTypes.biceptypes' as baz

param fooOrBar baz.fooOrBarType
param lbProperties baz.lbPropertiesType
param storageProperties baz.storagePropertiesType

resource foo 'SomeResourceTypeThatUsesParamsAbove' = { ... }

main.bicep

import 'br/public/types/myTypes.biceptypes' as bob
import 'br/public/providers/kubernetes:1.0.0'

module mod 'mod.bicep' = {
  name: 'mod'
  params: {
    storageProperties: bob.storageProps
    lbPropertiesType: bob.lbProps
    fooOrBar: 'bar'
  }
}

[brwilkinson]

comment to follow.

[jeskew]

Allowing the definition of user-defined types in a Bicep module (i.e. a .bicep file) creates as a consequence the problem that we are trying to solve in this proposal. Instead, I propose that we disallow defining user-defined types and constants in .bicep files and instead introduce a new file type .biceptypes (or an aesthetically equivalent name) to specify these kinds of symbols.

I think we would still need to deal with the case where an OCI artifact has the requisite layers to be both an extensibility provider and a source of constants. (If we were to introduce a new kind of artifact just for constants, an artifact could be all three of A) an extensibility provider, B) a template, and C) a constants source, which I think if anything makes this ambiguity worse.) Is the new artifact type meant to make export unnecessary (kind of like how .h files have no visibility modifiers, because if it's in a header file, it's meant to be public)?

[asilverman]

Is the new artifact type meant to make export unnecessary (kind of like how .h files have no visibility modifiers, because if it's in a header file, it's meant to be public)?

I believe so, export would not be necessary as any content inside a .biceptype is meant to be imported inside a .bicep/.bicepparam/.bicepdeploy.

[asilverman]

I think we would still need to deal with the case where an OCI artifact has the requisite layers to be both an extensibility provider and a source of constants.

With the current proposal, an extensibility provider is not able to define constants and user-defined types This limitation is unfortunate since an extensibility provider is a great candidate for their use. For example, I can imagine a potential GitHub extensibility provider wanting to define the structure of the GitHub REST API objects and also define some constants/enums for particular purpose too... After all these are also modeling the provider domain in a way similar to resources. If an ext. provider can define resource types, it follows that it should also define compositions of built-in types

[jeskew]

With the current proposal, an extensibility provider is not able to define constants and user-defined types This limitation is unfortunate since an extensibility provider is a great candidate for their use.

I think two things are being conflated here. The provider can define the shape of resource PUT requests it expects, and the Bicep compiler will provide validation of those domain objects, but resource body validation is always issued as non-blocking warnings. This is very different from how parameter/output validations (including decorators like @minValue/@minLength and user-defined types) are enforced; violating a constraint defined in the template will always cause a deployment to fail and, if we can tell at compile time that the deployment will fail, compilation will be blocked.

This difference in validation levels exists because provider resource shapes are a snapshot in time of an API contract that will be enforced at some point in the future by a system outside of ARM's control, whereas types defined in the template have deterministic behavior at deploy time that can always be known at compile time. With resource body validation, there is potential for drift between when a provider publishes its manifest and when the validation will be performed (even assuming the provider types were 100% at the time of publication, which is not always the case). I would think we would discourage extensibility providers from supplying "user-defined types" (which exist solely to provide blocking validation of template parameters and outputs) and instead encourage them to validate the bodies of resource PUT requests.

For example, I can imagine a potential GitHub extensibility provider wanting to define the structure of the GitHub REST API objects and also define some constants/enums for particular purpose too...

If an extensibility provider did want to define some template authoring helpers in the form of vars or UDFs, I don't think it's too much to ask them to publish two artifacts (or an extra layer within the artifact that defines the extensibility provider). We could end up in a situation where a user needs to include and import from the same source, but I think that's preferable to making import unnecessarily enlarge a compiled template (if the user wanted to hook up the extensibility provider but not use any of the provider's constants) or slow down a deployment or require the deployer to have registered an extensibility provider they won't be using (if the template author wanted to use shared constants but not deploy any resources via that provider). The proposal for include gives the template author a lot of control over which symbols to pull in (which is missing from the import statement) and would not necessitate a new .biceptypes file format like you're arguing import would, since I don't think anyone would assume that include {foo} from 'bar.bicep' would deploy the baz resource described in 'bar.bicep'.

After all these are also modeling the provider domain in a way similar to resources. If an ext. provider can define resource types, it follows that it should also define compositions of built-in types

I don't see how that follows. You're proposing that Bicep introduce a single abstraction ("types") that covers both ARM parameter validation and ARM resource provider routing, but I would argue that any single abstraction will need to be leaky and will thus cause more confusion than just keeping the concepts distinct. A "resource type" in ARM is just HTTP routing information: the fact that a resource has a type of "Microsoft.Storage/storageAccounts@2023-12-31" tells ARM where to send the PUT request to deploy a resource of that kind (as well as what value to use for the apiVersion query string parameter). Bicep goes above and beyond what the ARM engine does by providing advisory validation of the resource body, but this is done on a best effort basis because of the potential for drift between when Bicep captured the expected input schema for a given resource kind and when the resource provider will validate its input prior to starting a deployment (and because provider schemas are never 100% accurate). Bicep can't reasonably block compilation of a template that may very well deploy successfully, but it can and does block compilation of templates whose in-template validation logic ensures that deployment will fail (e.g., output foo string = 21).

---

I think one other issue that's factoring into this discussion is that the current import statement may be confusingly named. If it were instead provider 'kubernetes@1.0.0', I think that would better communicate what the statement actually does: give ARM an additional set of routes for HTTP PUT requests. register or using might also be good options. It would also be nice to use a noun as the keyword (like we do for param, var, resource, etc). (I like the idea of using a verb for include since it's something the compiler does rather than something the compiled template has, though.)

[WhitWaldo]

@asilverman Unfortunately, the output of a Bicep build includes all the custom type information (e.g. it's not purely a development-time artifact like an interface in TypeScript), which means that all this custom type data would ultimately have to be bundled alongside any of the modules in one way or another anyway. The creation of a special file type just to contain them would still necessitate treating it as some sort of module and then handling import from it like any other file, so I'm not sure what the benefit would be.

@jeskew I think your idea of changing the keyword for providers to provider from import is a really good one as it's a better fit to purpose in contrast to the importation of types throughout.

[jeskew]

I updated the issue description to reflect recent discussions and have moved the alternative proposals from comments to sections of the main description.

[asilverman]

I just had a thought over night I wanted to share. I think we could potentially allow user-defined types declaration in .bicep files and only restrict sharing of there user-defined types by means of a .biceptypes/.bicept file. That way, if you are looking to prototype something you can easily do so within your .bicep file and once its mature you can move your models to a .biceptypes with the beneficial side-effect of having clear separation of code which aligns with industry best-practice, that is, to keep the resource model separated from the behavior (more info)

[asilverman]

Unfortunately, the output of a Bicep build includes all the custom type information (e.g. it's not purely a development-time artifact like an interface in TypeScript), which means that all this custom type data would ultimately have to be bundled alongside any of the modules in one way or another anyway.

If you think of an ARM-JSON as an intermediate language that will eventually be read only by the machine, then having the behavior be as you describe is a non-issue.

In fact, I think it reinforces the claim that the definitions should be handled as a separate construct. Consider how importing them from a .bicep file would affect the compilation. If I understand correctly, the author will use the import gesture to partially load symbols from the .bicep module and then those symbols would be bundled alongside any of the modules regardless since I don't thing that ARM-JSON supports 'scope closures'. I may be wrong about that so please correct me if that is the case.

I'm not sure what the benefit would be.

The benefits as I see them are:

1. Enhanced readability: A novice reader of a .bicep deployment file will likely struggle to reconcile cognitively why a .bicep file is being both used in the import gesture and the module gesture. The behavior assumes that the reader is familiar with the nuance that only the types will be imported and not the resources which is a big assumption and may lead users to confusion as well as additional customer requests for clarification increasing on-call noise to signal ratios.
2. The ability for providers to share user-defined types: The current design boxes in the ability for providers to share user-defined types. I can imagine that an extensibility provider may want to do so for various reasons, including the need to enforce a particular structure on the properties of resources it exposes as well as a need to specify constants. By segregating user-defined types to a special file then the provider can bundle that file into its provider package. Its important to consider providers expose types only as types.json at the moment and so perhaps a workaround to this matter is to allow specifying user-defined types in the types.json serialization protocol too.
3. The practice of separating resource model from behavior is a software best practice. Since we are talking about sharing symbols for user-defined-types then it makes sense to follow best practices when publishing them, those practices being: clear separation of concerns and decoupling model from behavior.

There may be more benefits as well that are not immediately apparent:

1. Reduction of on-call load as a result of less questions/confusion resulting from the uncertainty while reading a .bicep file that has bot the import and module gestures used on the same bicep module.

That said, its just my personal experience and opinion and some of these benefits so I am trying to keep an open mind about it

[jeskew]

@asilverman Could you open a separate issue for the .biceptypes proposal? The question merits its own discussion but has limited bearing on what gets decided here. Whatever file/artifact type is used to declare sharable values, we will need some way to import/include them in other files.

[brwilkinson]

Can we link to the other 2 (or 3?) proposals from here?

[jeskew]

@brwilkinson The issue description has three proposals (1 main proposal and two alternatives); these aren't written up anywhere else.

There are some references in the discussion to the user-defined functions (UDFs) proposal (#9239) and the Bicep extensibility proposal (#3565).