Generator requires generator and template source

[ShadowLordAlpha]

So I was working on an experiment to try and get both #16 and #100 working but the generator needs its sources. I assume that this is for the time stamp of when the file was last changed so that it can skip extra work but is that all its for?

I'm asking because I have each binding split into different sub projects and the current system is designed so that everything is assumed to be in the same place.

[Spasi]

I assume that this is for the time stamp of when the file was last changed so that it can skip extra work but is that all its for?

Yes, incremental Kotlin compilation in IDEA + checking time stamps was the only way to have a decent workflow while working on templates.

The current scheme also uses the Kotlin sources to discover and run the template classes using reflection (using Class.forName("$packageName.${it.nameWithoutExtension.upperCaseFirst}Kt")).

A few comments on your experiment:

• First of all, thanks for spending time on improving LWJGL and having the courage to mess with Kotlin. :)
• Being able to programmatically configure the build would be very useful indeed. Ant works fine for now, but I don't like how the definitions for each binding are scattered in several places/scripts. So if there's no way to refactor that into something clean, we'll have to find a better, scriptable, solution. With that said, I don't think working on Gradle build #16 is worth the effort. I've had to deal with a lot of horrible stuff while working on LWJGL, but nothing comes close to Gradle.
• The work on Separation of documentation from the main generator #165 should take place before anything else. I haven't shared my exact thoughts on it yet, but let me just say that I've been thinking about converting all templates to something that isn't Kotlin code. I can now say that writing the generator in Kotlin was worth it, moving away from Java interfaces + annotations was worth it, using Kotlin for the templates themselves was not worth it. The compilation overhead and IDE issues when working on big templates (of which we have many) have become too painful to deal with and I'm willing to sacrifice the (partial) type-safety and other Kotlin goodies we have now. Also, we've had virtually no contributions to binding templates so far... I think moving away from Kotlin will help with that.
• I don't think Modular design for build artifacts #100 is very important yet. With the current LWJGL design, most bindings have literally no native code at all. Anything you don't need from the LWJGL build you can take out easily: strip any packages you don't need from lwjgl.jar and don't include the shared libraries you don't use. The lwjgl shared library is quite small and won't grow that much.
• I created Modular design for build artifacts #100 to be able to easily work on random bindings, that have little or nothing to do with what LWJGL is usually used for and don't make sense in the default distribution. E.g. two years ago I created ODBC bindings with LWJGL 3 (which work fantastically well btw), or earlier this month I wanted to try LMDB and I had it up and running in LWJGL in an afternoon. Just like Add Oculus Platform SDK bindings #169, I don't see a problem with including all that in the LWJGL repo, as long as we have control of what is enabled by default and goes into the official distribution.

[ShadowLordAlpha]

(On a phone so quotes are hard)

• I MESS WITH ALL THE THINGS. But really I learned kotlin doing this so one more language I can say I know how to use.
• gradle isn't that bad (It can look ugly though if you set it up like my experiment is) though normally it looks much cleaner.
• if you move over to non compiled templates that would make almost everything easier.
• while I would agree with this people are lazy and it can cause problems if they remove something they shouldn't
• I agree but would say that makes it more important and if you rewrite the generator for template files and desperate documentation it would probably be a good time to introduce Modular design for build artifacts #100 as well

As a side note it took me 1 1/2 days to do it but the current generator will compile and generate the sources in my experiment (not on github yet as I'm on a bus) but it does work.

EDIT: code is now up on github

[ShadowLordAlpha]

@Spasi currently on my lwjgl3 fork there is a gradlebuild branch that is working with the current generator and can tell it to build or not to build parts though its still very simple. If you want to try it just open a command window and run the command gradlew generate and it will generate all the different source files though I still have a bit of work to do on it.

[Spasi]

Hey @ShadowLordAlpha, could you please rebase the gradlebuild branch on top of the current master and push it again? It's very hard to review your changes in the current state.

Btw, generated sources are ignored in the main repository because tiny changes in the generator can produce massive diffs in the generated code. We do track progress though, in a separate repository, here.

In other news, Kotlin 1.0.2-beta now supports incremental compilation in Gradle builds.

[ShadowLordAlpha]

Ya give me a bit to fully do it

[ShadowLordAlpha]

Btw, generated sources are ignored in the main repository because tiny changes in the generator can produce massive diffs in the generated code. We do track progress though, in a separate repository, here.

I know but they have to go somewhere.

In other news, Kotlin 1.0.2-beta now supports incremental compilation in Gradle builds.

decently sure that its 1.0.1-2 that supports it

[ShadowLordAlpha]

@Spasi ok the gradle-build branch should be what you want. To run it correctly open up a command window in the same folder and use the command gradlew run you will probably get errors from the missing java files so use gradlew copy to get those from the core source set to the main source set.

Some notes

• Most of the code was refactored to different areas but its all still one project and should be easy enough to navigate around. ask if you have any problems with it though it should be very similar to the current code.
• This gradle buildfile was only built to build and run the generator code so that is all it does.
• It will probably give you errors if you only have a JRE and not a JDK.
• I removed the signing and maven plugins for now as i have not used them and don't want to mess with them right now
• if something doesn't makes sense ask me though it should explain itself decently well
• it can build the lwjgl jar using the command gradlew jar though I haven't really worked on that part so it may not work properly or fully as expected right now

[Spasi]

decently sure that its 1.0.1-2 that supports it

According to the Change Log, you'll need an 1.0.2 build.

Most of the code was refactored to different areas

OK, I see the new structure, but I don't know why it was necessary. The current structure was chosen to match Gradle/Maven conventions.

This gradle buildfile was only built to build and run the generator code

Compiling and running the generator is simple enough and I already have a gradle build that does that (locally, haven't pushed it on github anywhere). The issue I had was making Gradle behave like the current Ant build, where inactive bindings are not included in the source path at all. Your build doesn't do that either. It's probably not a big deal now with incremental compilation, but it'd be nice to have.

Another thing I had done is use a master build.gradle under the root, then have a separate build.gradle for each module (under modules/core and modules/templates). Not sure if that made things unnecessarily complex and contributed to the trouble I had.

[ShadowLordAlpha]

According to the Change Log, you'll need an 1.0.2 build.

oh ok ya that is 1.0.2 though i think i was getting that with 1.0.1-2 though i could be wrong

OK, I see the new structure, but I don't know why it was necessary. The current structure was chosen to match Gradle/Maven conventions.

It technically wasn't needed however its the convention that gradle uses so you don't have to configure each plugin like the kotlin and java plugins though those two are not all that hard to compile and should make using some other plugins easier as no obscure configurations would be needed.

Compiling and running the generator is simple enough and I already have a gradle build that does that (locally, haven't pushed it on github anywhere). The issue I had was making Gradle behave like the current Ant build, where inactive bindings are not included in the source path at all. Your build doesn't do that either. It's probably not a big deal now with incremental compilation, but it'd be nice to have.

I can do that (I wrote this at 5 in the morning so I had to go to sleep) its set up so that it can be done. I just didn't do it. Anything else you need it to do that it currently does not do?

Another thing I had done is use a master build.gradle under the root, then have a separate build.gradle for each module (under modules/core and modules/templates). Not sure if that made things unnecessarily complex and contributed to the trouble I had

I find that doing that does makes things more complex as you have to keep track of the code in bunch of different places (I only ever use that for multiproject builds and even then I don't normally use it as if I keep everything in one file I can reuse a lot of variables most of the time).

[ShadowLordAlpha]

@Spasi done, inactive bindings should now be excluded from the generated code. This includes the kotlin files (both compiled and templates though this might need to be modified with a shift over to non-compiled templates) and core files. Be careful though as excluding some things may cause problems. Currently it should work based on the system arguments you gave generator before (they are in gradle.properties). It looks messy and can be cleaned up a lot though so its only showing how to do it really.