Perform generation of files using rust instead of node.js

[AiyionPrime]

While I see the benefit of easily generating the files using js,
maintaining a second language with its respective ecosystem just to generate files every once in a while appears to be an overkill.

I think after the reproducibility of the generator has been verified in

• ci: Regenerate and verify the status codes file #344
• ci: Regenerate and verify the supported messages file #347
• ci: Regenerate and verify the address space files #348
• ci: Regenerate and verify the types files #350
• ci: Regenerate and verify the node ids file #351
  we might implement a rust workspace which performs the generation instead.

Later we might split the generated code in another separate workspace, which would then not need to be compiled each and every time one changes something in this library.

Lastly using rust instead of js brings us one step further to resolving #224.

[einarmo]

Solving #224, I think, would mean creating two libraries, one for parsing the NodeSet2 file, and one for performing code-gen based on it.

That may be a decent approach for replacing the code-gen in general. Create one library that builds data structures based on the various OPC-UA files, and another that generates code based on them.

[evanjs]

This would be great
I actually had to work around this a while back and hacked it into my app's build.rs
But seeing it upstreamed would be great

Note: my own approach is very much not suited for direct contribution IMO
I mostly just "auto converted" from JS to RS and fixed a few things that stuck out

[einarmo]

Figured I'd mention I've started looking into this.

Just replacing the codegen is easy enough, but we'd want to take things a bit further. There are a few issues with the current approach:

• the .bsd file is deprecated, we should be using the .xsd file instead.
• We don't import Value, which is actually fairly important, since it contains all default values.

This means implementing parsing for the Opc.Ua.Types.xsd file, which actually means code generation based on xml schemas. I looked around but didn't find any library that suited our purposes well enough...

So yeah, I'm writing a limited library for generating code based on .xsd files. Hopefully we can then use the code generated from that to create code to generate code based on NodeSet2 files.

[AiyionPrime]

Oh. I did so as well. Currently I'm at the second of five? generators.
I started your suggestion and split parsing and templating.

I'm using askama for the latter, which works pretty well.

[AiyionPrime]

Ah; I started with the easy ones to assess askama. csv is working very well, xsd I haven't touched yet.

[einarmo]

Ah, okay, I've been just using syn directly, with quote for the codegen. See https://github.com/einarmo/opcua/tree/codegen/opcua-codegen for something that uses the .bsd file, though as mentioned I want to replace that.

[einarmo]

Turns out the bsd file, while marked as obsolete, contains info not in the xsd file, so we'd need to somehow bridge the two if we want proper handling of values in the nodeset2 file...

[locka99]

I used nodejs as a convenience since it is easy to turn around generation of files. If it were to move to Rust, then it would probably be a lot messier to code up without the JS backtick notation for variable substitution. If there were a way to do it in Rust (e.g. some kind of substitution macros that I'm not aware of) it is still preferable to do it as a separate stage that generates files which are checked in since it is a lot easier for IDEs to process the files and to view them in GitHub.

I don't think it would be a good idea to incorporate this into build.rs without separate standalone tools.

[einarmo]

If we do it right we make it so that we get a library that can be invoked both from the command line and from a build script. I agree with you that it's nice to have concrete types that can be viewed and read. Doing it this way though, we make it possible to use it from build.rs without needing to interface with node.

Code gen in rust is quite nice, in my experience, in that you can generate the code while parsing it. You use something like syn, which lets you have a method that returns, for example, the syntax tree of a struct. You can use quote for substitution.

One major advantage to note is that since we're writing code to parse the xml files in rust, and convert them to rust data structures, we can reuse that to load nodeset files at runtime. This is pretty much a necessity if we want to support much larger NodeSet2 files, I've seen ones with 100k+ nodes, which you definitely couldn't import using code generation.

[einarmo]

I've got something that works now with NodeSet2 files in rust, I even got importing Value to work (though I definitely realize now why you didn't do that the first time around, it took easily another 1000 lines of code...). The way it works now is that you get an empty type that implements the trait

pub trait NodeSetImport {
    fn register_namespaces(&self, namespaces: &mut NodeSetNamespaceMapper) -> Vec<String>;

    fn load<'a>(self, namespaces: &'a NodeSetNamespaceMapper) -> impl Iterator<Item = ImportedItem> + 'a;
}

pub struct ImportedItem {
    pub node: NodeType,
    pub references: Vec<ImportedReference>,
}

This is generic, so it can be used to import OPC-UA types without writing them to an AddressSpace, which is nice, and it is also a trait, so you could implement this on a type that only loads the nodes at runtime.