Mavenization to support development in NetBeans IDE

[karussell]

Hi Shay,

I've struggled a bit with git but now I have hopefully the correctly changed files to compile and test ElasticSearch from within NetBeans as there is no working gradle plugin available. Please ignore all the commits :) just have a look in the changed files. If depencencies changed, all the pom's can be automatically regenerated via gradlew and the script pkg/mavenize.sh

Important: I had to move 2 config files originally placed under java now under resources (to be maven compatible) - does this affect your build?

What I didn't understand: why is sigar not included in the gradle deps?

Regards,
Peter.

[karussell]

I'll check if I can avoid moving the config/text files (as there are more missing).

[dadoonet]

I just love this pull request !
Just wondering if your aim is to have a pom.xml file always available in github repository or if developpers will have to generate it ?

Thanks
David.

[karussell]

Thanks :)

Well the aim is to always have a pom.xml in the repository :)

But for new dependencies or new versions I wanted to automate the updating process of this pom.xml from the gradle file. Thats why the mavenize.sh (some parts are not necessary anymore e.g. moving the configs)

[kimchy]

What I am hoping to eventually do is to move ann the plugins to their own repo, and then, there will be just one module in elasticsearch, which will simplify things a lot. Back then, the reason why I gave up on maven was its poor support for multi module projects.

[karussell]

Ok. I would really go the gradle route if there would be better IDE support as with maven I really miss the flexibility of a real programming language in my build file (e.g. for deploying).

So, do you think that it would be wiser to close this issue here and just keep it as a reference for others? Or will you go the (probably not that clear) route and add the poms to the project?

BTW: the last commits are my latest trials to submit several pull requests. I made the mistake that this pull request is from my master and now I cannot do further changes to it without popping up here :/

[kimchy]

I think we can go a different route. I don't use any plugin in intellij for gradle, the intellij configuration is committed to the repo, and you can simply clone and import and it "just works". We can do something similar with netbeans, though now, with intellij community edition, the more people moving to use it, the better :)

[karussell]

though now, with intellij community edition

ah, didn't know that it has also the gradle support. But the IntelliJ-only-one-project-open-at-a-time thing is too limiting for me (e.g. work vs. hobby projects ;))

We can do something similar with netbeans

but this would mean: either use a normal java project (done via ant under the hood) and manually adding the dependencies ... or using the pom.xml

[dadoonet]

the reason why I gave up on maven was its poor support for multi module projects.

I'm afraid I have to disagree. I use every day maven for complex projects with several sub-modules and I don't have any bad feedback.
Yes, that's true that you can't easily do complex things, but should we ?
Yes, Gradle is more flexible. I see it as a mix between ant and maven.
I love maven because it gives me some guides and rules (even constraints). I love maven because when I come from a project to another one, I'm "at home". That's the same for all my coworkers.

Also, I think it's a really good idea to have gradle and/or maven and be able to setup our IDE with simple commands like

We don't know what IDE developpers use. Perhaps some use only Notepad and the command line :-(
IMO, I'm not sure it's a good idea to maintain settings for each IDE (I use eclipse - not sure we have to maintain a pom.xml, gradle settings, Intellij settings, netbean settings and .project file aka eclipse settings)...

I started to mavenized ES project some months ago but I was stucked with the jarjar plugin. I was starting to get some nice results but I didn't have enough spare time to go deeper.

If I understood Peter's commit, I can see that we have to manually create the es-jarjar jar (with gradle) before being able to play with maven, right ? (I have to confess that I didn't try to get your source code and play with).

[karussell]

Perhaps some use only Notepad and the command line :-(

well then gradle is also fine. also Shay should use the things which he is most comfortable with :)

netbean settings

there is no such thing

before being able to play with maven, right?

yes, but this is just a java programm to create the jarjar. not entirely sure if it is necessary at all (Shay uses this to avoid version conflicts with guice, netty, ...)

(I have to confess that I didn't try to get your source code and play with).

not that difficult. just create the jarjar and use the (already generated) pom.xml of this pull request ...

[jprante]

Hey, I did port several versions of Elasticsearch to Maven and Netbeans, I started several months ago. It's not complete with all the plugins though, just what I needed. The jarjar thing was reset to the original dependencies, and yes, I rebuild all dependencies from source in Maven by the way (well, almost everything).
Finally, besides building up a source-complete local repo in Subversion und Maven, I put the whole beast into Jenkins and Sonar with Findbugs for some fancy analysis.
If you like to have a look, visit http://demeter.hbz-nrw.de/sonar
I put up Lucene in Sonar for you as a bonus ;-) Cheers, Jörg

[karussell]

I did port several versions of Elasticsearch to Maven and Netbeans

Do you need to do this for every version or do you have a script?

The jarjar thing was reset to the original dependencies

what do you mean here?

[jprante]

Jarjar helps to ship a single jar and removes external jar dependencies by rewriting package names. In Elasticsearch, jarjar moves packages from e.g. Guice), Joda, GNU Trove etc. into something like org.elasticsearch.common.xyz. Shay tries to minimize external dependencies to obtain more reliability, I think. But I dropped the jarjar preprocessing and reset the original external dependencies in Maven's pom.xml.

The split into Maven tree-style src/{main,test}/java and src/{main,test}/resources has also to be done. I use a mixture of diff/patch und a simple find/replace script to apply the version changes. It can't be fully automated though. I use Maven tests (surefire) to validate the result. It takes some hours to edit a few number of patches for generating a working Elasticsearch version, it depends on the volume of Shay's changes.

[karussell]

But I dropped the jarjar preprocessing and reset the original external dependencies in Maven's pom.xml

So you really changed every class in ElasticSearch to use the original dependency?

The split into Maven tree-style src/{main,test}/java and src/{main,test}/resources has also to be done

not necessarily, it can be done via configuring this in the pom.xml

It takes some hours to edit a few number of patches for generating a working Elasticsearch version,
it depends on the volume of Shay's changes.

Hmmh, I think the changes are too intensive ... where can I view a resulting pom.xml in your repo? (didn't find it via browsing)

[jprante]

I changed the package names to the original ones in all the java sources, yes.
The open source repo is at http://demeter.hbz-nrw.de/svn/os
Current ES parent pom is http://demeter.hbz-nrw.de/svn/os/elasticsearch/release/0.18.4/pom.xml
I need an approval system that our ES based search app is truely using open source licenses in all its dependencies, therefore I decided to collect and rebuild the code from groundup, together with the licenses. And, more technically, I gathered the whole sources also to have them ready for fixing them, when rebuilding the complete ES software stack using OpenJDK 1.7.0.

[karussell]

Thx Shay :)

Kudos for all the refactoring work!