From 74f851dbda45b0172d0f59b10f3faf87b98ac51c Mon Sep 17 00:00:00 2001 From: Max Haughton Date: Wed, 11 Nov 2020 11:59:44 +0000 Subject: [PATCH 1/4] First draft of DIP to introduce __ATTRIBUTE__ --- DIPs/1NNN-MHH.md | 242 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 242 insertions(+) create mode 100644 DIPs/1NNN-MHH.md diff --git a/DIPs/1NNN-MHH.md b/DIPs/1NNN-MHH.md new file mode 100644 index 000000000..8fb6bb7e7 --- /dev/null +++ b/DIPs/1NNN-MHH.md @@ -0,0 +1,242 @@ +# Introduce `__ATTRIBUTE__` + +| Field | Value | +|-----------------|-----------------------------------------------------------------| +| DIP: | (number/id -- assigned by DIP Manager) | +| Review Count: | 0 (edited by DIP Manager) | +| Author: | Max Haughton mh2410@[universityofbathdomain] | +| Implementation: | github.com/maxhaton/dmd (too buggy for a PR as of writing) | +| Status: | Will be set by the DIP manager (e.g. "Approved" or "Rejected") | + +## Abstract +This DIP allows user-defined attributes to access the declarations onto which they are attached. This is achieved by the introduction of a new `SpecialKeyword` "\_\_ATTRIBUTE\_\_" to the grammar. When used as a default +initializer this shall be resolved to a `string[]` containing the fully qualified names of the declarations, if any, the expression's parent UDA declaration is attached to. + +Note that the exact name (i.e. "ATTRIBUTE") is easily changed. +## Contents +- [Let UDAs see what they are attached to](#) + - [Abstract](#abstract) + - [Contents](#contents) + - [Rationale](#rationale) + - [Prior Work](#prior-work) + - [Description](#description) + - [Reference](#reference) + - [Copyright & License](#copyright--license) + - [Reviews](#reviews) + +## Rationale +For a UDA to do work in D, it currently must be accessed via the scope in which the declarations it is attached to are declared. For most uses this is ideal: for example when serializing a data structure using a UDA is an efficient and maintainable way of specifying the semantics of how the given data structure is to be serialized. +### A simple example of a UDA - Serializing an exam result +```D +struct Serialize { + string impl; + this(string forWhom) + { + impl = forWhom; + } +} +struct ExamResult { + //The student's name is needed in all reports + @Serialize("ParentReport") @Serialize("InternalReport") + string studentName; + //The examiner's name is only needed for internal moderation + @Serialize("InternalReport") + string examinerName; + //Same as the first + @Serialize("ParentReport") @Serialize("InternalReport") + float overallScore; + //Don't serialize this + string dbEntry; +} +``` + +Clearly in this case there is a strictly top-down topology to the usage of these user-defined attributes, whenever this structure is serialized D's metaprogramming facilities makes it trivial to *iterate* (emphasis added for later relevance) over `ExamResult`'s members and look for any attributes relevant to our little library. +### A less simple example +There are, however, situations where a UDA is highly useful but there does not exist as clean of a hierarchy as in the previous example. + +For example, an attribute will often be attached to a - in effect - free standing symbol like a `unittest`. +```D +enum runMyTest; + +@runMyTest unittest { + +} +``` +Since most of these declarations are at or near module scope, we can simply iterate over *all* the `unittest`s looking for one's matching our UDA. Although not technically required, it is still common to find these tests in a pre-compilation step - or by manually setting the test runner to work using a `mixin` statement. + +This pattern is fine for `unittest`s: When used, there are usually many of them, and when there are not many the potential for a performance hit is mitigated by the compiler providing a list of `unittest`s within a given scope. + +### The behaviour to be enabled by this DIP. +There are, however, situations where this pattern is less than ideal. Some UDAs may be used to declare things which are both sparse and not `unittest`s. + +If we setup a mechanism to get these UDA-ed symbols - assuming we know where they are i.e. a recursive search is currently not possible but scopes containing UDAs can be declared using a different UDA - we still have to iterate over *every* declaration in a given scope looking for those with our desired set of attributes. For a sparse declaration - that is, if we are looking for a few declarations out of a several thousand line file - this is not an efficient way of doing things. + +For example, this DIP is motivated specifically by a desire to be able to declare benchmarks in the following manner (without needing the user to `mixin` anything): + +```D +@Benchmark!(SomeInformationAboutTheBenchmark) +float dotp(float[3] vec1, float[3] vec2) pure +{ + return vec1[0] * vec2[0] + vec1[1] * vec2[1] + vec1[2] * vec2[2] +} +//The library can then do what it wants with dotp, be that schedule it to run or add it to a table at compile time +``` + +This DIP proposes a simple solution to this problem: Let the UDA see sideways, that is, let it see what it is attached to. Rather than being some paradigm shift in how UDAs are used, this should simply be viewed as having the same effect of adding syntactic sugar for the following construct: +```D + module home; + enum MyUDA; + template HandleTheUDA(alias handleThis) {/* impl */} + @MyUDA void widget() {} + //Ugly + mixin HandleTheUDA!handleThis; + //Slow, we don't want to search the entire module just for one UDA + mixin HandleTheUDA!home; +``` +i.e. With this DIP, MyUDA can be declared as a template that can perform the `mixin` itself. This does not add any new dragons or side effects beyond what the original construct could do. +## Prior Work +### A pattern that already does the job +The following pattern can be used within a UDA to find what said UDA is attached to. +```D +//Module is needed for obvious reasons, the line parameter makes it unique for most uses +template FinderUDA(string name, string m = __MODULE__, int l = __LINE__) +{ + import std.format; + import std.traits; + enum FinderUDA; + mixin(format!"alias mod = %s;"(m)); + //Your implementation goes here + pragma(msg, getSymbolsByUDA!(mod, FinderUDA)); +} + +@FinderUDA!"Hello" +void wow() +{ + import std.stdio; + writeln("WOW!"); +} +``` +In this case it prints "tuple(wow)". + +This pattern has obvious flaws: Unless told where to look recursively, it can only work with symbols at module scope, the attribute declaration/s must be on separate lines to be uniquely identified, and more subtly in a big file we now have to search through every declaration that the UDA *could* be attached to. The compiler already has this information, let's use it. + +### How the patterns enabled by this DIP are done in C++ +Thanks to the simplicity of textual preprocessing, C++ can enable this pattern using a macro. However, this is extremely error-prone so a common way of achieving everything mentioned so far in this article is to have a separate compilation step to collect and use information from the source code. + +For example, Epic Games' Unreal Engine uses the following syntax to interface between C++ and "Blueprints" +```C++ +class AGameActor : public AActor +{ + GENERATED_BODY() + public: + UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = Properties) + FString Name; + UFUNCTION(BlueprintCallable, Category = Properties) + FString ToString(); +}; +``` +The game then inserts hijacks the compile to read the (in our terms) user-defined attributes from the header file, however this not only hurts compilation times but is also fairly antithetical to the "D way" of doing things. +## Description +This DIP simply proposes adding to the language a new *SpecialKeyword* `__ATTRIBUTE__` to the grammar. +```diff +SpecialKeyword: + __FILE__ + __FILE_FULL_PATH__ + __MODULE__ + __LINE__ + __FUNCTION__ + __PRETTY_FUNCTION__ ++ __ATTRIBUTE__ +``` +It will always resolve to a string array literal. +```D +void main(string[] args) +{ + const attr = __ATTRIBUTE__; + pragma(msg, typeof(attr), attr); //"const(string[]) and []" +} +``` +This literal shall be empty unless `__ATTRIBUTE__` is used as a default initializer, in which case it shall be resolved to either to be either empty or an array literal of the fully qualified names of all declarations a UDA is attached to, if and only if (subject to existing default initializer resolution) it is resolved to an expression within said *UserDefinedAttribute* + +### Some specific examples: +A templated struct +```D +module testmodule; +struct Test(string name, string[] attr = __ATTRIBUTE__) +{ + pragma(msg, name, " says: ", attr); + this(int l) {/*Do work*/} +} +@Test!"name"(1) +int echo(int x) +{ + return x; +} +//"name says: ["testmodule.echo"]" +``` +A simple function +```D +module testmodule; +auto just(string[] at = __ATTRIBUTE__) +{ + return at; +} + +@(just()) +int x, y, z; + +pragma(msg, pragma(msg, __traits(getAttributes, x)[0])); +//"["testmodule.x", "testmodule.y", "testmodule.z"]" +``` +A class +```D +class wow { + string[] cont; + this(string[] attr = __ATTRIBUTE__) + { + cont = attr; + } +} +@(new wow) +int cheese; +pragma(msg, __traits(getAttributes, cheese)[0]); +//"wow(["testmodule.cheese"])" +``` +Finally, eliminating a `mixin`. +```D +template runThisFunction(string name, string[] attrs = __ATTRIBUTE__) +{ + pragma(msg, "DRT", attrs); + enum runThisFunction; + + shared static this() + { + import std.stdio; + static foreach(at; attrs) { + mixin("alias theFunc = " ~ at ~ ";"); + writef!"%s is running %s"(name, at); + theFunc(); + } + + } +} +@(runThisFunction!"Darth Vader") +void runMe() +{ + import std.stdio; + writeln("dlang"); +} +//Eliminates having to write @runThisFunction then using a mixin to actually do the work +``` +## Reference +[The Unreal Engine property system](https://www.unrealengine.com/en-US/blog/unreal-property-system-reflection) + + +## Copyright & License +Copyright (c) 2020 by the D Language Foundation + +Licensed under [Creative Commons Zero 1.0](https://creativecommons.org/publicdomain/zero/1.0/legalcode.txt) + +## Reviews +The DIP Manager will supplement this section with a summary of each review stage +of the DIP process beyond the Draft Review. From 58095f5d1bafc9e0f7ec4224bb5de9f5ff15f9fc Mon Sep 17 00:00:00 2001 From: Max Haughton Date: Wed, 11 Nov 2020 20:36:07 +0000 Subject: [PATCH 2/4] Incorporated C&C from bolpat --- DIPs/1NNN-MHH.md | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/DIPs/1NNN-MHH.md b/DIPs/1NNN-MHH.md index 8fb6bb7e7..8ddaf6237 100644 --- a/DIPs/1NNN-MHH.md +++ b/DIPs/1NNN-MHH.md @@ -9,8 +9,7 @@ | Status: | Will be set by the DIP manager (e.g. "Approved" or "Rejected") | ## Abstract -This DIP allows user-defined attributes to access the declarations onto which they are attached. This is achieved by the introduction of a new `SpecialKeyword` "\_\_ATTRIBUTE\_\_" to the grammar. When used as a default -initializer this shall be resolved to a `string[]` containing the fully qualified names of the declarations, if any, the expression's parent UDA declaration is attached to. +D's user-defined attributes are already great, this DIP simple change to make them better: Let the UDA see sideways, that is, let it see what it is attached to. Letting them do this let's them eliminate many tasks the preprocessor can do that D cannot. This is achieved by the introduction of a new `SpecialKeyword` "\_\_ATTRIBUTE\_\_" to the grammar. When used as a default initializer this shall be resolved to a `string[]` containing the fully qualified names of the declarations, if any, the expression's parent UDA declaration is attached to. Note that the exact name (i.e. "ATTRIBUTE") is easily changed. ## Contents @@ -88,12 +87,16 @@ This DIP proposes a simple solution to this problem: Let the UDA see sideways, t enum MyUDA; template HandleTheUDA(alias handleThis) {/* impl */} @MyUDA void widget() {} - //Ugly + mixin HandleTheUDA!handleThis; - //Slow, we don't want to search the entire module just for one UDA + //^A mixin per symbol is ugly + mixin HandleTheUDA!home; + //^A searching mixin has the potential to be very slowwe don't want to search the entire module just for one UDA ``` -i.e. With this DIP, MyUDA can be declared as a template that can perform the `mixin` itself. This does not add any new dragons or side effects beyond what the original construct could do. +i.e. This DIP enables the library writer to create a UDA that can manage the per-symbol `mixin` itself. In translating our example to this new idiom *MyUDA* would go from being an `enum` to a template as the code contained within *HandleTheUDA* is now done at the UDA instead. When the library is consumed, however, the API would be the same due to the use of default parameters (other than the lack of `mixin`s). + +This does not add any new dragons or side effects beyond what the original construct could do. ## Prior Work ### A pattern that already does the job The following pattern can be used within a UDA to find what said UDA is attached to. @@ -101,20 +104,15 @@ The following pattern can be used within a UDA to find what said UDA is attached //Module is needed for obvious reasons, the line parameter makes it unique for most uses template FinderUDA(string name, string m = __MODULE__, int l = __LINE__) { - import std.format; import std.traits; enum FinderUDA; - mixin(format!"alias mod = %s;"(m)); + mixin("alias mod = " ~ m ~ ";"); //Your implementation goes here pragma(msg, getSymbolsByUDA!(mod, FinderUDA)); } @FinderUDA!"Hello" -void wow() -{ - import std.stdio; - writeln("WOW!"); -} +void wow() { } ``` In this case it prints "tuple(wow)". @@ -148,7 +146,7 @@ SpecialKeyword: __PRETTY_FUNCTION__ + __ATTRIBUTE__ ``` -It will always resolve to a string array literal. +It will always resolve to a string array literal - A user-defined attribute may be attached to one *or more* declarations, necessitating the use of `string[]` rather than `string` (See examples below). ```D void main(string[] args) { @@ -156,7 +154,7 @@ void main(string[] args) pragma(msg, typeof(attr), attr); //"const(string[]) and []" } ``` -This literal shall be empty unless `__ATTRIBUTE__` is used as a default initializer, in which case it shall be resolved to either to be either empty or an array literal of the fully qualified names of all declarations a UDA is attached to, if and only if (subject to existing default initializer resolution) it is resolved to an expression within said *UserDefinedAttribute* +This literal shall be empty unless `__ATTRIBUTE__` is used as a default initializer, in which case it shall be resolved to either to be either empty or an array literal of the fully qualified names of all declarations a UDA is attached to, if and only if (subject to existing default initializer resolution behaviour) it is resolved to an expression within said *UserDefinedAttribute* ### Some specific examples: A templated struct @@ -213,9 +211,11 @@ template runThisFunction(string name, string[] attrs = __ATTRIBUTE__) { import std.stdio; static foreach(at; attrs) { - mixin("alias theFunc = " ~ at ~ ";"); - writef!"%s is running %s"(name, at); - theFunc(); + { + mixin("alias theFunc = " ~ at ~ ";"); + writef!"%s is running %s"(name, at); + theFunc(); + } } } From 25230d01d23d4496652e6fecb6458b7f3033f547 Mon Sep 17 00:00:00 2001 From: Max Haughton Date: Wed, 11 Nov 2020 20:39:08 +0000 Subject: [PATCH 3/4] grammar --- DIPs/1NNN-MHH.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/DIPs/1NNN-MHH.md b/DIPs/1NNN-MHH.md index 8ddaf6237..ce6d45935 100644 --- a/DIPs/1NNN-MHH.md +++ b/DIPs/1NNN-MHH.md @@ -9,7 +9,7 @@ | Status: | Will be set by the DIP manager (e.g. "Approved" or "Rejected") | ## Abstract -D's user-defined attributes are already great, this DIP simple change to make them better: Let the UDA see sideways, that is, let it see what it is attached to. Letting them do this let's them eliminate many tasks the preprocessor can do that D cannot. This is achieved by the introduction of a new `SpecialKeyword` "\_\_ATTRIBUTE\_\_" to the grammar. When used as a default initializer this shall be resolved to a `string[]` containing the fully qualified names of the declarations, if any, the expression's parent UDA declaration is attached to. +D's user-defined attributes are already great, this DIP simple change to make them better: Let the UDA see sideways, that is, let it see what it is attached to. Letting them do this lets them eliminate many tasks the preprocessor can do that D cannot. This is achieved by the introduction of a new `SpecialKeyword` "\_\_ATTRIBUTE\_\_" to the grammar. When used as a default initializer this shall be resolved to a `string[]` containing the fully qualified names of the declarations, if any, the expression's parent UDA declaration is attached to. Note that the exact name (i.e. "ATTRIBUTE") is easily changed. ## Contents @@ -66,7 +66,7 @@ Since most of these declarations are at or near module scope, we can simply iter This pattern is fine for `unittest`s: When used, there are usually many of them, and when there are not many the potential for a performance hit is mitigated by the compiler providing a list of `unittest`s within a given scope. ### The behaviour to be enabled by this DIP. -There are, however, situations where this pattern is less than ideal. Some UDAs may be used to declare things which are both sparse and not `unittest`s. +There are, however, scenarios where this pattern is less than ideal. Some UDAs maybe highly expressive and yet both sparse and not applied to `unittest`s, but the cost of using them may add up in a large D program. If we setup a mechanism to get these UDA-ed symbols - assuming we know where they are i.e. a recursive search is currently not possible but scopes containing UDAs can be declared using a different UDA - we still have to iterate over *every* declaration in a given scope looking for those with our desired set of attributes. For a sparse declaration - that is, if we are looking for a few declarations out of a several thousand line file - this is not an efficient way of doing things. @@ -92,7 +92,7 @@ This DIP proposes a simple solution to this problem: Let the UDA see sideways, t //^A mixin per symbol is ugly mixin HandleTheUDA!home; - //^A searching mixin has the potential to be very slowwe don't want to search the entire module just for one UDA + //^A searching mixin has the potential to be very slow; we don't want to search the entire module just for one UDA ``` i.e. This DIP enables the library writer to create a UDA that can manage the per-symbol `mixin` itself. In translating our example to this new idiom *MyUDA* would go from being an `enum` to a template as the code contained within *HandleTheUDA* is now done at the UDA instead. When the library is consumed, however, the API would be the same due to the use of default parameters (other than the lack of `mixin`s). From ea212104a9721780469095dddbe575ac14184bd0 Mon Sep 17 00:00:00 2001 From: Max Haughton Date: Mon, 16 Nov 2020 12:24:59 +0000 Subject: [PATCH 4/4] Added draft status, changed __ATTRIBUTE__ to the more descriptive __UDA_ATTACHED_TO_DECLS__ --- DIPs/1NNN-MHH.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/DIPs/1NNN-MHH.md b/DIPs/1NNN-MHH.md index ce6d45935..f4e721bc4 100644 --- a/DIPs/1NNN-MHH.md +++ b/DIPs/1NNN-MHH.md @@ -1,4 +1,4 @@ -# Introduce `__ATTRIBUTE__` +# Introduce `__UDA_ATTACHED_TO_DECLS__` | Field | Value | |-----------------|-----------------------------------------------------------------| @@ -6,12 +6,12 @@ | Review Count: | 0 (edited by DIP Manager) | | Author: | Max Haughton mh2410@[universityofbathdomain] | | Implementation: | github.com/maxhaton/dmd (too buggy for a PR as of writing) | -| Status: | Will be set by the DIP manager (e.g. "Approved" or "Rejected") | +| Status: | Draft | ## Abstract -D's user-defined attributes are already great, this DIP simple change to make them better: Let the UDA see sideways, that is, let it see what it is attached to. Letting them do this lets them eliminate many tasks the preprocessor can do that D cannot. This is achieved by the introduction of a new `SpecialKeyword` "\_\_ATTRIBUTE\_\_" to the grammar. When used as a default initializer this shall be resolved to a `string[]` containing the fully qualified names of the declarations, if any, the expression's parent UDA declaration is attached to. +D's user-defined attributes are already great, this DIP simple change to make them better: Let the UDA see sideways, that is, let it see what it is attached to. Letting them do this lets them eliminate many tasks the preprocessor can do that D cannot. This is achieved by the introduction of a new `SpecialKeyword` "\_\_UDA_ATTACHED_TO_DECLS\_\_" to the grammar. When used as a default initializer this shall be resolved to a `string[]` containing the fully qualified names of the declarations, if any, the expression's parent UDA declaration is attached to. -Note that the exact name (i.e. "ATTRIBUTE") is easily changed. +Note that the exact name (i.e. "UDA_ATTACHED_TO_DECLS") is easily changed. ## Contents - [Let UDAs see what they are attached to](#) - [Abstract](#abstract) @@ -135,7 +135,7 @@ class AGameActor : public AActor ``` The game then inserts hijacks the compile to read the (in our terms) user-defined attributes from the header file, however this not only hurts compilation times but is also fairly antithetical to the "D way" of doing things. ## Description -This DIP simply proposes adding to the language a new *SpecialKeyword* `__ATTRIBUTE__` to the grammar. +This DIP simply proposes adding to the language a new *SpecialKeyword* `__UDA_ATTACHED_TO_DECLS__` to the grammar. ```diff SpecialKeyword: __FILE__ @@ -144,23 +144,23 @@ SpecialKeyword: __LINE__ __FUNCTION__ __PRETTY_FUNCTION__ -+ __ATTRIBUTE__ ++ __UDA_ATTACHED_TO_DECLS__ ``` It will always resolve to a string array literal - A user-defined attribute may be attached to one *or more* declarations, necessitating the use of `string[]` rather than `string` (See examples below). ```D void main(string[] args) { - const attr = __ATTRIBUTE__; + const attr = __UDA_ATTACHED_TO_DECLS__; pragma(msg, typeof(attr), attr); //"const(string[]) and []" } ``` -This literal shall be empty unless `__ATTRIBUTE__` is used as a default initializer, in which case it shall be resolved to either to be either empty or an array literal of the fully qualified names of all declarations a UDA is attached to, if and only if (subject to existing default initializer resolution behaviour) it is resolved to an expression within said *UserDefinedAttribute* +This literal shall be empty unless `__UDA_ATTACHED_TO_DECLS__` is used as a default initializer, in which case it shall be resolved to either to be either empty or an array literal of the fully qualified names of all declarations a UDA is attached to, if and only if (subject to existing default initializer resolution behaviour) it is resolved to an expression within said *UserDefinedAttribute* ### Some specific examples: A templated struct ```D module testmodule; -struct Test(string name, string[] attr = __ATTRIBUTE__) +struct Test(string name, string[] attr = __UDA_ATTACHED_TO_DECLS__) { pragma(msg, name, " says: ", attr); this(int l) {/*Do work*/} @@ -175,7 +175,7 @@ int echo(int x) A simple function ```D module testmodule; -auto just(string[] at = __ATTRIBUTE__) +auto just(string[] at = __UDA_ATTACHED_TO_DECLS__) { return at; } @@ -190,7 +190,7 @@ A class ```D class wow { string[] cont; - this(string[] attr = __ATTRIBUTE__) + this(string[] attr = __UDA_ATTACHED_TO_DECLS__) { cont = attr; } @@ -202,7 +202,7 @@ pragma(msg, __traits(getAttributes, cheese)[0]); ``` Finally, eliminating a `mixin`. ```D -template runThisFunction(string name, string[] attrs = __ATTRIBUTE__) +template runThisFunction(string name, string[] attrs = __UDA_ATTACHED_TO_DECLS__) { pragma(msg, "DRT", attrs); enum runThisFunction;