Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test plan for "module initializers" feature #40500

Closed
30 tasks done
jcouv opened this issue Dec 19, 2019 · 40 comments
Closed
30 tasks done

Test plan for "module initializers" feature #40500

jcouv opened this issue Dec 19, 2019 · 40 comments
Labels
Area-Compilers Documentation Test Test failures in roslyn-CI Test-Gap Describes a specific feature or scenario that does not have test coverage
Milestone

Comments

@jcouv
Copy link
Member

jcouv commented Dec 19, 2019

Championed issue: dotnet/csharplang#2608
Specification: https://github.com/dotnet/csharplang/blob/master/proposals/module-initializers.md

  • LangVer
  • missing module init type: [ModuleInit(typeof(Missing))]
  • module init type has a use-site error [ModuleInit(typeof(A))] where A can't be fully loaded (A defined in a library, but it's base type is referenced in another library which is not referenced in current/client library)
  • test some netmodule scenarios (adding multiple modules within an assembly)
  • attribute on local functions
  • attribute on static and non-static methods
  • attribute on methods with parameters
  • attribute on non-void methods
  • attribute on generic methods
  • attribute on methods directly or indirectly in a generic type, method is generic and not generic
  • attribute on methods inaccessible on the module level
  • attribute on method inside an added .NET module compiled from IL. Expect the method to be not executed.
  • attribute on multiple methods in source. Expect methods are executed in lexical order. The spec should clarify the order.
  • different methods decorated with different attributes (extern aliases). Should make no difference, all methods should be executed.
  • methods decorated with the attribute using constructor other than default are ignored.
  • multiple attributes on the same method (change attribute definition to allow multiple applications)
  • attributes on legacy partial methods: on definition, on implementation, on both (change attribute definition to allow multiple applications)
  • attributes on new-style partial methods: on definition, on implementation, on both (change attribute definition to allow multiple applications)
  • attributes on a static constructor
  • attributes on methods in an interface
  • attributes on non-method declarations (change attribute definition to allow), should be ignored.
  • attribute on conditional method
  • attribute is conditional
  • attribute on an obsolete method
  • Error when attribute is used in VB
  • No warning when module initializer method initializes a static field with a non-nullable reference type seems to be a difficult analysis to reason about correctly. Test plan for "module initializers" feature #40500 (comment)
  • Expected semantics for static int field = default; when module initializer also writes to field
    • the optimization should not be a problem because the static constructor will run before the module initializer can write anything.

BCL

LDM

FYI @jnm2

@jcouv jcouv added this to the Compiler.Next milestone Dec 19, 2019
@jnm2
Copy link
Contributor

jnm2 commented Dec 19, 2019

Based on dotnet/csharplang#2608, I was also thinking there would be tests that the obsoletion error would be suppressed when targeting C# 9.

What about when you're using C# 8 and declaring your own ModuleInitializerAttribute and you leave out the ObsoleteAttribute? Should the compiler error when you apply that to the module even though there is no obsoletion error? Also, what if you declare your own ModuleInitializerAttribute with incorrect attribute usage and apply it e.g. to the assembly instead of to the module, or apply it to the module twice?

@gafter gafter added Test-Gap Describes a specific feature or scenario that does not have test coverage Test Test failures in roslyn-CI labels Jan 2, 2020
@jnm2
Copy link
Contributor

jnm2 commented Mar 28, 2020

@jcouv Should I submit a PR to add LanguageVersion.CSharp9 yet?

@jnm2
Copy link
Contributor

jnm2 commented Mar 29, 2020

@jcouv I have a success case working! Next I'll add tests and implement diagnostics. master...jnm2:module_initializer

@jcouv
Copy link
Member Author

jcouv commented Mar 29, 2020

@jnm2 It sounds like we need to confirm the language proposal with LDM. I think it’s scheduled mid April.
Tagging @gafter

@jnm2
Copy link
Contributor

jnm2 commented Mar 29, 2020

@jcouv Should I wait till then to implement diagnostics?

@jcouv
Copy link
Member Author

jcouv commented Mar 29, 2020

Probably safer.

@gafter
Copy link
Member

gafter commented Mar 30, 2020

Although we have a draft spec for the feature, there are at least four or five different viable approaches to solving the problem in the language, and I don't have confidence that the draft spec is the approach we'll end up selecting.

@jnm2
Copy link
Contributor

jnm2 commented Apr 8, 2020

Should I proceed to implement the conclusion section of https://github.com/dotnet/csharplang/blob/master/meetings/2020/LDM-2020-04-08.md#module-initializers, or is it still too soon?

@jcouv
Copy link
Member Author

jcouv commented Apr 8, 2020

@jnm2 The agreement on design seemed pretty firm (all static methods marked with an attribute contribute to module initialization). I think it's okay to go ahead.

Tagging @RikkiGibson @jaredpar @gafter as FYI.

@jnm2
Copy link
Contributor

jnm2 commented Apr 8, 2020

Implementation questions:

  • Would the static method be permitted to be private or protected?
  • Would the method be required to return void?
  • There's nothing wrong with extern or anything not mentioned thus far, right?
  • Should applying the attribute to a finalizer be disallowed? (AttributeUsage can't help with this)
  • What technique would you recommend to keep the order deterministic?
  • Do I use the LanguageVersion.Preview flag and write diagnostic messages as though C# 9 is not a thing?

Which of these error situations should be ignored and how should diagnostic IDs be assigned to the rest?

  • The attribute is applied to an instance method (AttributeUsage can't help with this)
  • The attribute is applied to a method that has parameters
    • Should there be a more specific error message if the attribute is applied to an accessor or operator? (AttributeUsage can't help with this)
  • The attribute is defined by me (in source or externally) because I'm using C# 9 and compiling for .NET Core 3.1:
    • I don't add AttributeUsage to my definition and proceed to apply it to non-method targets like classes/assemblies/static constructors
    • I don't add AttributeUsage to my definition and proceed to apply it more than once to the same target
    • I add AttributeUsage with Inherited = true and for some reason expect that this would work
    • I add members to the definition and proceed to apply it using constructor or named arguments
  • The attribute is applied in a project with langversion < 9

Just being dumb now if I wasn't before:

  • What does the runtime do if I define the ModuleInitializerAttribute myself and apply it to a static method within ModuleInitializerAttribute? I'm guessing this wouldn't trigger anything special.

@RikkiGibson
Copy link
Contributor

  • Would the static method be permitted to be private or protected?

Probably not, because we will need to emit a call to it in the module initializer method, which as far as I can tell is just a special method on a special class.

Would the method be required to return void?

Yes, it it also not permitted to have parameters

There's nothing wrong with extern or anything not mentioned thus far, right?

I don't see anything that should block this, but it is also pretty easy to wrap an extern method in some static method that you then mark with [ModuleInitializer].

What technique would you recommend to keep the order deterministic?

Don't know exactly where this lives in the compiler, but the examining the behavior for static fields in partial classes is probably a good starting point.

Do I use the LanguageVersion.Preview flag and write diagnostic messages as though C# 9 is not a thing?

Yes. I believe you add a member to the MessageID enum for the feature and then you modify the switch statement that indicates the required language version for the feature.

Which of these error situations should be ignored and how should diagnostic IDs be assigned to the rest?

  • The attribute is applied to a method that has parameters
  • The attribute is applied to a method that has a return value

Error on these.

The attribute is defined by me in source or IL because I'm using C# 9 and compiling for .NET Core 3.1:

No error on these. We generally accept well-known attributes that are user-declared, as long as they match the shape to the extent we need. This is likely to be how your tests will work as well.

The attribute is applied to an instance method (AttributeUsage can't help with this)

Error on this.

I don't add AttributeUsage to my definition and proceed to apply it to non-method targets like classes/assemblies/static constructors

It's okay to let AttributeUsage help you automatically where it can and add special cases to the compiler just to fill in the gaps. For a ModuleInitializerAttribute placed on a non-method target, I recommend silently ignoring the attribute.

  • I don't add AttributeUsage to my definition and proceed to apply it more than once to the same target

I recommend silently ignoring the redundant attributes.

I add AttributeUsage with Inherited = true and for some reason expect that this would work

I don't see how this can affect anything with static methods.

I add members to the definition and proceed to apply it using constructor or named arguments

Only usage of well-known attribute constructors should affect compiler behavior. Therefore you should silently ignore ModuleInitializerAttribute if it is using a constructor with an unknown signature.

The attribute is applied in a project with langversion < 9

I think we should, for the scenario that you are using an old target framework and old language version with a new SDK and you try defining the attribute manually.

Just being dumb now if I wasn't before:

  • What does the runtime do if I define the ModuleInitializerAttribute myself and apply it to a static method within ModuleInitializerAttribute? I'm guessing this wouldn't trigger anything special.

There's no problem with doing this. I would expect it to add a call to the static method within the generated module initializer. This would be a good test.

@RikkiGibson
Copy link
Contributor

Sorry if some of the questions got out of sync while I was writing. Let me know if you need further clarification on anything.

@gafter
Copy link
Member

gafter commented Apr 9, 2020

Would the static method be permitted to be private or protected?

We discussed this briefly but I don't think we answered it. There are pros and cons to permitting them

  • Pro: permits using program-appropriate protection to ensure that the method cannot be used for anything else, e.g. called multiple times.
  • Con: slightly more complex code gen. When the method is inaccessible, the compiler would have to emit a synthetic helper method with internal accessibility to perform the call. If the method is in a nested type, more than one such helper method may be required:
class A
{
    private class B
    {
        [ModuleInitializer]
        private static void M() { ... }
    }
}

would have to translate to something like

void __ModuleInitializer() { A.<helper>(); } // generated
class A
{
    internal static void <helper>() { B.<helper>(); } // generated
    private class B
    {
        internal static void <helper>() { B.M(); } // generated
        [ModuleInitializer]
        private static void M() { ... }
    }
}

@jnm2
Copy link
Contributor

jnm2 commented Apr 9, 2020

@gafter Is that something that would be safe to ban (allocating a diagnostic ID to it) for the time being?

Hey, just thought of two more cases:

  • The attribute is applied within a generic type
  • The attribute is applied to a generic method

@jcouv
Copy link
Member Author

jcouv commented Apr 9, 2020

@jnm2 I would assume only accessible methods can be used for now (simpler). We'll confirm with LDM by email and share the answer back.

@gafter
Copy link
Member

gafter commented Apr 9, 2020

I agree, looking at my notes I think we did decide that the method has to be accessible at the module level (effectively internal). The programmer would have to use some other mechanism to get the effect of private (e.g. mark the method Obsolete).

@jnm2
Copy link
Contributor

jnm2 commented Apr 11, 2020

I'm ready for feedback on #43281.

@jnm2
Copy link
Contributor

jnm2 commented Apr 12, 2020

What does "calling in a deterministic order" mean? Deterministic across which variables? Source member reordering, source file reordering, moving members between partial classes, or simply compiler version changes?

@jnm2
Copy link
Contributor

jnm2 commented Apr 12, 2020

These are all the new diagnostic IDs I'm planning to add. Should any be combined?

        ERR_ModuleInitializerMethodMustBeInternalOrPublic = 8793,
        ERR_ModuleInitializerMethodMustBeStatic = 8794,
        ERR_ModuleInitializerMethodMustNotHaveParameters = 8795,
        ERR_ModuleInitializerMethodMustReturnVoid = 8796,
        ERR_ModuleInitializerMethodMustNotBeGeneric = 8797,
        ERR_ModuleInitializerMethodMustNotBeContainedInGenericType = 8798,

@jnm2
Copy link
Contributor

jnm2 commented Apr 12, 2020

I think I'll have to change ERR_ModuleInitializerMethodMustBeInternalOrPublic to ERR_ModuleInitializerMethodMustBeAccessibleOutsideTopLevelType:

Module initializer method 'M' must be accessible outside top-level type 'C'

internal class C
{
    private class Nested
    {
        [ModuleInitializer]
        internal static void M() { }
    }
}

@jnm2
Copy link
Contributor

jnm2 commented Apr 12, 2020

Should there be some warning for [ModuleInitializer] on a static/nonstatic local function?

@RikkiGibson
Copy link
Contributor

Should there be some warning for [ModuleInitializer] on a static/nonstatic local function?

I suggest erroring on ModuleInitializer on a local function because the local function will not be accessible.

I think I'll have to change ERR_ModuleInitializerMethodMustBeInternalOrPublic to ERR_ModuleInitializerMethodMustBeAccessibleOutsideTopLevelType:

Sounds good.

These are all the new diagnostic IDs I'm planning to add. Should any be combined?

It's dealer's choice on this one, but consider combining 8795 and 8796, as well as 8797 and 8798.

What does "calling in a deterministic order" mean? Deterministic across which variables? Source member reordering, source file reordering, moving members between partial classes, or simply compiler version changes?

It just means that when we compile with the same set of inputs, we should produce the calls in the same order every time. Once you make changes to the inputs we may call the module initializers in a different order.

I suggest trying to do file+source position ordering, similar to how static initializers in partial classes work. Let me know if you need more specific guidance and I can try to look more deeply into how this works.

@jnm2
Copy link
Contributor

jnm2 commented Apr 12, 2020

Right now the same check that is causing local functions to be ignored is also what is causing operators/constructors/finalizers/accessors to be ignored. Should I produce an error for local functions and nothing else, or should I produce an error for more of these? (All valid 'method' attribute targets.)

Ah, so the variable to guard against with deterministic ordering would be concurrency during compilation.

@jcouv
Copy link
Member Author

jcouv commented Apr 12, 2020

I think of determinism as “run the same command with same inputs, get the same outputs”. Changing the order of inputs is changing an input.

Ah, so the variable to guard against with deterministic ordering would be concurrency during compilation.

Yes

@jaredpar
Copy link
Member

Should there be some warning for [ModuleInitializer] on a static/nonstatic local function?

I suggest erroring on ModuleInitializer on a local function because the local function will not be accessible.

Agree. Even a static local function is not a item that can be directly invoked from outside the containing function. This should apply to module initializers as well.

It just means that when we compile with the same set of inputs, we should produce the calls in the same order every time. Once you make changes to the inputs we may call the module initializers in a different order.

I think we should draw lessons here from partial types. These have similarities to module initializers in that they allow for partial member initializers to be spread out among the compilation. The compiler though will always order the field initializers by file order on the command line and top to bottom order within the same file. Would treat module initializers in the same way unless there was a compelling reason to do otherwise.

@jnm2
Copy link
Contributor

jnm2 commented Apr 13, 2020

Just to clarify, the question is whether to ignore [ModuleInitializer] on local functions entirely or to error. I'm currently ignoring them entirely because that's what fell out of the code I used to ignore [ModuleInitializer] entirely for operators, accessors, and finalizers per discussion above. (Early exit if targetSymbol.MethodKind != MethodKind.Ordinary.)

I'm wondering if I should special-case local functions so that an error is produced while the other targets are still ignored, or if I should just produce an error for every target where MethodKind != Ordinary.

I'm not sure, but I might have misunderstood @RikkiGibson here. I read this as saying [ModuleInitializer] should be silently ignored on operators, finalizers, and accessors (all of which are allowed by AttributeTargets.Method, but none of which are considered C# method syntax):

For a ModuleInitializerAttribute placed on a non-method target, I recommend silently ignoring the attribute.

@RikkiGibson
Copy link
Contributor

I was trying to describe cases where AttributeTargets would normally cause an error due to being placed on e.g. a type and not a method. No need to try and protect the user from themselves if they declare the attribute class with an unexpected AttributeTargets argument.

Since AttributeTargets can't disallow usage on operators, finalizers, accessors, local functions, etc, let's have a diagnostic for when [ModuleInitializer] is placed on the wrong kinds of methods.

@jnm2
Copy link
Contributor

jnm2 commented Apr 13, 2020

Thank you. I really appreciate your patience!

@jnm2
Copy link
Contributor

jnm2 commented Apr 25, 2020

@jcouv #43281 is about to merge which completes this item in your list:

  • LangVer (if this refers to diagnostics)

[edit: oops, two of the items are part of the second PR; removed]

What do you think about adding these items?

  • Error when attribute is used in VB
  • No warning when module initializer method initializes a static field with a non-nullable reference type

@jcouv
Copy link
Member Author

jcouv commented Apr 26, 2020

@jnm2 Added those two bullets. Thanks

@jnm2
Copy link
Contributor

jnm2 commented May 2, 2020

@jcouv Now that the language proposal, runtime spec, and initial implementation PR are merged, I opened dotnet/runtime#35749 as specified in your test plan above.

@RikkiGibson
Copy link
Contributor

@jnm2 this one could be a bit of a pain: "No warning when module initializer method initializes a static field with a non-nullable reference type"

The compiler today expects that a field will be initialized in the same type it is declared in, either in an explicit constructor or with a field initializer. But with ModuleInitializerAttribute you could initialize an accessible static field in any type. I don't think we actually have a precedent for flow analyzing multiple methods to find out the initialization state of multiple types.

We could potentially have special handling for a module initializer that is within the type that declares the field, treating it as running after the initializers/static constructor (because if the module initializer is really just a method on that type, its static constructor will run before the method). But people may get confused regardless if they have good reason to initialize the field from some other type.

I think the thing to do here may be to ask that people initialize their non-nullable reference static fields with null!, since the writes or even the .cctor on the type can often get optimized out.

@jnm2
Copy link
Contributor

jnm2 commented May 6, 2020

@RikkiGibson That makes sense to me.

/cc @YairHalberstadt who brought up the scenario on Gitter

@jnm2
Copy link
Contributor

jnm2 commented May 12, 2020

@jcouv These are implemented now that #43301 is merged:

  • attribute on local functions
  • attribute on static and non-static methods

@valdisiljuconoks
Copy link

Hi, looks like a great feature. Was just wondering - whether you are planning to add some sort of dependency feature between modules (like execute this init procedure only after that module has been initialized)?

Thanks!

@weltkante
Copy link
Contributor

weltkante commented Jul 17, 2020

execute this init procedure only after that module has been initialized

@valdisiljuconoks That kind of dependency does not make sense, a module needs to be initialized before code from that module is executed, so delaying initialization until some condition is true is not possible (you wouldn't be able to execute dependent code)

Module initialization however should already be lazy and only done when you try to execute something within the module (I don't know which exact conditions trigger module initialization). If you have a dependency that some other module be initialized before yours, even if you don't execute code from it, then you can easily encode this through a call to RuntimeHelpers.RunModuleConstructor (you can get the module reference by referencing a type in the assembly, or by following and resolving your own modules assembly references)

@jnm2
Copy link
Contributor

jnm2 commented Jul 17, 2020

https://github.com/dotnet/runtime/blob/master/docs/design/specs/Ecma-335-Augments.md#module-initializer:

A module initializer is executed at, or sometime before, first access to any static field or first invocation of any method defined in the module.

Notably, a module initializer is not executed via Assembly.GetTypes(). This means that the assembly that GetTypes is being called on can't use module initializers to hook AssemblyResolve in time to avoid type load exceptions. See dotnet/csharplang#2608 (comment) and my comment below which links demos of this for .NET Core and .NET Framework.

@valdisiljuconoks
Copy link

valdisiljuconoks commented Jul 18, 2020

Probably described poorly. What I was looking for - is to try to see if module initializers could replace infrastructure that we are using from underlying CMS platform. They have initialization engine that is responsible for calling found init modules across all assemblies in app domain (done via scanning). Each init module could declare dependency - I can be called only after other init module has been called. That other module could be located in different assembly. You specify dependency via [ModuleDependency(typeof(OtherModule)].

Was just wondering - whether this feature could be used instead.

@jnm2
Copy link
Contributor

jnm2 commented Jul 30, 2020

@RikkiGibson @jcouv The more I think about it, the less I like the idea of allowing ModuleInitializerAttribute to be emitted as a custom attribute. Leaving it on the method makes it look like it has meaning to the runtime. Is there any reason in favor of leaving it? It doesn't answer the question "is this MethodInfo called from the module initializer" because that depends on the compiler version that the method had been compiled with.

@jcouv
Copy link
Member Author

jcouv commented Mar 17, 2023

This test plan was fully checked. I'll go ahead and close it.
Tagging @AlekseyTs @jnm2 @RikkiGibson as FYI

@jcouv jcouv closed this as completed Mar 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Compilers Documentation Test Test failures in roslyn-CI Test-Gap Describes a specific feature or scenario that does not have test coverage
Projects
None yet
Development

No branches or pull requests

7 participants