Base compileTime on dotnet-script CLI tool instead of Roslyn scripting

[dadhi]

Knowing that the current compileTime directive is based on Roslyn scripting (C# interactive - CSI), I have started to look into other scripting limitations.

What I found is a modern scripting alternative - dotnet-script CLI tool https://github.com/filipw/dotnet-script. Here is the detailed article of the C# scripting tools comparison https://itnext.io/hitchhikers-guide-to-the-c-scripting-13e45f753af9

As for dotnet-script, it is cross-platform, works with .NET Core v3.1+, allows referencing of NuGet packages, allows packing, and referencing other scripts from the NuGet.

It also seems to support more broad C# sub-set including C# 8, for comparison, CSI even does not support using static, not talking about extension methods.

Additionally, there are goodies to access the script path:

    public static string GetScriptPath([CallerFilePath] string path = null) => path; 
    public static string GetScriptFolder([CallerFilePath] string path = null) => Path.GetDirectoryName(path);

So my question, is it possible to consider dotnet-script for the next version LeMP templating?

[qwertie]

I wasn't aware of this tool before, but I just downloaded it and gave its interactive mode dotnet script a try. Not surprisingly, I got this result:

> namespace X {}
(1,1): error CS7021: Cannot declare namespace in script code

Next I tried writing a full script with a namespace:

#!/usr/bin/env dotnet-script
namespace X {
  public class Y {
    public static void Main(string[] a) {
      Console.WriteLine("Hello, world!");
    }
  }
}

No luck:

C:\Temp>dotnet-script test.csx
C:\Temp\test.csx(4,24): warning CS7022: The entry point of the program is global script code; ignoring 'Y.Main(string[])' entry point.
C:\Temp\test.csx(2,1): error CS7021: Cannot declare namespace in script code
...

There are a couple of key reasons I used interactive mode. One is that the input comes as a series of separate invocations of the compileTime and precompute macros. In particular, precompute might be used dozens/hundreds of times in a single file, and I assumed C# interactive would be more efficient in this scenario than, say, compiling a new assembly for every expression evaluated. In any case, you can define things in one block and use them in a later block (the reverse doesn't work - compileTime { new X(); } compileTime { class X {} } produces an error) which made the interactive mode a good fit for LeMP.

A second reason is that C# interactive allows mixing of instructions and declarations (e.g. Console.WriteLine("Hi"); class X { } is valid in csi and not normal C#), which made it super easy to use because I did not need to write any code to transform code like that into normal non-interactive C#. Perhaps this second factor explains why dotnet-script doesn't support namespaces in csx files.

I doubt I will find the time to do this myself in the immediate future, but if you really need features like namespaces, you could write a macro that would compile a set of source files (*.cs and even *.ecs) into an assembly using Roslyn or csc or msbuild, and then add that assembly as a reference to the current ecs file. I imagine a macro like

// Line 7 of Example.ecs:
compileTimeReference("SourceFile1.cs", "SourceFile2.ecs", 
    "Reference1.dll", "Reference2.dll", { namespace X { class Y {} } });

This, of course, would allow SourceFile1.cs and SourceFile2.ecs to contain namespaces and extension methods, and it wouldn't be hard to also support inline blocks of code that contain namespaces and extension methods, as shown.

compileTimeReference("Reference.dll"); should probably be equivalent to compileTime { ##reference("Reference.dll"); }, but if you specify source files or source code, it should compile a DLL. For example, the macro could set up a Roslyn CSharpCompilation object, compile a DLL called C:...\Temp\Example.cs.Line7.dll and then replace itself with compileTime { ##reference("C:\...\Temp\Example.cs.Line7.dll"); } to reference the DLL.

[dadhi]

Thanks for looking.. I am not interested in namespaces so much, but rather in extension methods and more simple folder references, and other goodies e.g. using static.

I will check your suggestions. Thanks for your time!

[qwertie]

I just checked again. dotnet-script matches C# interactive (and LeMP): it does not support extension methods but it does support using static. dotnet-script does have unique features in its version of #r that are not in C# interactive or LeMP.

[dadhi]

it does not support extension methods

Hmm, this is the full script which works for me:

public static string Bang(this string x) => x switch { { Length : 6 } => x + "!!!", _ => "nah" };
var sailor = "Sailor".Select(char.ToUpper).Aggregate(new StringBuilder(), (s, c) => s.Append(c)).ToString().Bang();
WriteLine($"Hi, {sailor}");

dotnet-script does have unique features in its version of #r

Yep, this is a win for quick prototyping or trying things (and probably much more - because you can wrap script in .exe or publish to nuget):

#r "nuget: FsCheck, 2.14.3"
using FsCheck;
Prop.ForAll<int[]>(x => x.Reverse().Reverse().SequenceEqual(x)).QuickCheck();

[qwertie]

Ahh, I misunderstood the situation. C# interactive actually does support extension methods if they are not within a class - they must be at the top level. So I will change the next version of LeMP to (at least) allow extension methods at the top level.

[dadhi]

@qwertie

Currently, there is a trend to provide minimal (or single file) solutions to the historically big entreprizy problems. I think it is a good opportunity to demonstrate LeMP. Or at least test it features on something small and complete.
What do you think?

Here are two threads:

• https://twitter.com/shanselman/status/1281856685657616384?s=20
• https://twitter.com/dodyg/status/1284749334756589575?s=19

Expanding

Maybe for removing the solution and project out of the picture, generating the dotnet-script .csx file from the .ecs is enough. At least for now.

The first obvious Ecs thing is warping the multiple using lines into a single (.bla, .foo)

Others? TBD

[qwertie]

I have published LeMP 2.8.2 which allows extension methods directly inside compileTime (still not inside classes inside compileTime).

Are they really adding top-level statements to the main language (not C# interactive)? Ever since C# 1.0 I always wanted top-level functions. I wouldn't complain if they add top-level statements... but what would happen if multiple source files have top-level statements?

I've always thought that the build system should be based on the same syntax as the programming language itself... this makes the most sense if the syntax is very generic, like LES, but imagine if, in C#, top-level statements were actually commands like BuildAssembly("Foo.dll", listOfFiles, options). Perhaps there should be "library" source files, with .cs extension, and then "build system" source files like .csproj, but with C# syntax. Of course, some care would have to be taken in the design to make a friendly graphical editor possible for project files...

There are a ton of angles that can be can taken to make software development better - whether the problems are "enterprizey" or not. The people at Future of Coding tend to focus on highly experimental "visual" software development prototypes, for example, or interesting debugging experiences; I tend to be interested more in the theoretical side of "how to make a better programming language' - and there are so many ways to improve languages, but C# and .NET have a lot of historical baggage that I wish I could break free from. MS of course has thousands of times more resources than one unemployed developer (=me) and thanks to .NET Core they are finally breaking free from some of it - like it was a pleasant surprise that they added non-nullable reference types (though unfortunately they designed Nullable<T> in a way that is fundamentally incompatible with generics), and ditto for Span<T>. And now, finally, I hear that Covariant Return Types are on the table. I've wanted that since C# 1.0, too.

What EC#/LeMP really needs is more and better tooling, I think. Not having IntelliSense is a pretty big limitation, among other things. But, I kind of need a stronger indication that people want to use LeMP before I invest a huge amount of free time in tooling. And even then, I may still prefer to work on an entirely new language (which of course would also include LeMP) - I'd probably make it compatible with C# syntax, and maybe other popular syntaxes (TypeScript?) but it might not be mainly targeted at .NET, nor would C# be the main syntax. Plus, I think cross-platform support is important so I've been thinking about supporting VS Code, but on the other hand VS is the most popular IDE for C#. But VS Code and VS have totally unrelated APIs, so supporting both would be irritating work.

[dadhi]

@qwertie

Thanks for release!

Are they really adding top-level statements to the main language (not C# interactive)?

Yes.. and while it is a bonus feature, honestly I would like them to spend the time on something else. Probably I don't like that features in the latest C# releases are not designing to play together. Seems like the feature inclusion solely decided on its creation effort, plus ASP team request, plus marketing. I would love to see more coherence like it was with Linq, which was a bigger thing built out of the smaller things being useful on its own.
Regarding the top-level statements itself I would prefer your take on namespace without braces. It may be even expanded further to public class Namespace.MyClass; without braces. Especially useful for static classes aka modules (as F# takes it).

[dadhi]

I've always thought that the build system should be based on the same syntax as the programming language itself... this makes the most sense if the syntax is very generic, like LES, but imagine if, in C#, top-level statements were actually commands like BuildAssembly("Foo.dll", listOfFiles, options)

Agree. I was liked the idea of build system, packaging, and the whole infra done in the same language. It both minimizes the cognitive load and simplifies the system, given that you are already have tooling for the language. This is done in other spaces where it was not an afterthought, Elm for instance takes that in a Web space including everything you need for dev and deploy. Jai (in closed beta) has a very rich compile time capabilities - you may run a graphical game at compile-time written in the same language. I highly recommend to watch the series of videos of the Jai author, Jonathan Blow https://www.youtube.com/playlist?list=PLmV5I2fxaiCKfxMBrNsU1kgKJXD3PkyxO

Back to C# I would like my build script to be as simple as possible and better is not existent. But sometimes you don't have a choice. Then it would be cool to not learn a new language and even better if I can develop and debug the build system the same as the rest if the program. There is a Nuke-build which has a similar ideas. But I did not like its UX. Neither I like the Cake. Some time ago I was looking at Nake which has a pleasant simple API but never have a chance to try it out.
What I think that the space for build systems in C# is still vast and lacks of simple solutions. Not sure for the LeMP applicability here at the moment, at least until it has a dotnet-script like model of a single file doing everything and ability to reference other things (other scripts, packages, e.g. ready-to-use tools).

Btw, the is another closely related space of configuration-as-a-program (example is Dhall - https://github.com/dhall-lang/dhall-lang) and much bigger cloud-infra-as-a-program (see Pulumi - https://www.pulumi.com/docs/intro/languages/dotnet).

[dadhi]

What EC#/LeMP really needs is more and better tooling, I think. Not having IntelliSense is a pretty big limitation, among other things. But, I kind of need a stronger indication that people want to use LeMP before I invest a huge amount of free time in tooling.

It is understandable. Documentation, tooling and support is rarely an interesting thing to do in your free time.

I will just add my feedback regarding the lack of tooling, as I see it in LeMP.

I would've want intellisence as well, syntax highlighting and debugging.. in this order. Important thing here is the usage environment too.

• In VS Code I am happy with the C# syntax highlighting that I may apply on ecs file as easy as selecting the respective Language Mode from the Command Pallette.
• I don't need a debugging as long as I can "println" some info while compiling. Actually, I am not sure how to output something while in LeMP - so maybe you help.
• Intellisense is a most problematic. In VS Code you need to implement the extension talking to Language Server via Language Server Protocol.. which is likely a hard to implement (at least for me). But having LSP may be a reusable trait for other integrations.
  There is also a much simplier partial solution to have a snippets from Ecs (e.g. for macros) - https://code.visualstudio.com/docs/editor/userdefinedsnippets.
  I have an idea that Ecs may somehow leverage the OmniSharp LS. It works for scripting Today. I think it worth to try as a much smaller effort. At least I will play with renaming the .ecs to .cs and get back to you with results.

Regarding Visual Studio - it is still a biggest player and the begemoth :). But Today you have an escape path and not the only one, there is JB Rider as well. I have a feeling that going forward (with fast start and cross-plat in mind) VS Code will play more and more prominent role, especially when trying the new things

[qwertie]

Good point, Console.WriteLine isn't going to do anything interesting inside Visual Studio (or is it? I don't know, but I'm reluctant to call Console.SetOut because it would affect the entire process.) So I have made a code change for the next release that will enable you to write warning and error messages as follows:

compileTime {
    MessageSink.Default.Warning("location1", "Message to {0}", "you");
    MessageSink.Default.Error("location2", "Note to {0}", "self");
}

... but for now you can use precompute() to output strings into the output file. Just remember that if you precompute inside compileTime, the expression is precomputed before any of the compileTime block is evaluated and the result is not written to the output unless you change it to compileTimeAndRuntime.