[Dev How To] Running ONDEX and the Integrator from Eclipse

There are two ways to run/debug ONDEX code from Eclipse (other IDEs should be similar to what we're going to describe here): one is by starting an ONDEX application (or Maven tests) from out of Eclipse, using the line commands and the Java debugger protocol, the other is by starting an application (or a JUnit test) from within Eclipse, using a run/debug configuration.

Settings

We will assume you have the M2E Eclipse plug-in for Maven and that you've used it to import the ONDEX-Related projects you are interested in by the steps:

1. Clone from GitHub to your local directory. From now on, ondex-knet-builder will refer to the directory where you put this clone.
2. Eclipse -> File -> Import -> Existing Maven Projects ->
3. In the last step there, we recommend to select all (ie, Maven main project and sub-modules), to put everything under a working set (there are many modules, better to keep things tidy), to resolve namespace projects (advanced options) and to use a 'name template' like 'knet::[artifactId]', to assign Eclipse names consistently and avoid clashes with other projects (e.g., other KNetMiner versions/copies). In the following, we will assume you have used this name pattern, so we will make references like knet::installer (which corresponds to <artifactId>installer</artifactId>).

Debugging command line executions

ONDEX

1. Build ONDEX from the command line. The cleanest way to do so is to cd to the root folder corresponding to the repository cloned above (e.g. $HOME/ondex-knet-builder) and issue mvn clean install.
2. A quicker way is cd into installer and run mvn clean install from there. However, this will pick dependencies from your local Maven repository and, if not found there, from our ONDEX repositories and other external repositories. So, you might miss the change you've just made.
3. Once you have ONDEX built, cd ondex-knet-builder/installer/target and unzip the .zip that you should have in this directory.
4. Once done, you should have an ONDEX distribution in target/ondex/. runme.sh should be able to launch it and point to the config, plugins and other files in this distribution. To start debugging it, change the last line in runme.sh, the one that has the java command, this way:

java -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005 \
     -Xmx$MEMORY -Dondex.dir=$DATA -Dovtk.dir=$OVTK_DATA -Dplugin.scan.lib=false -classpath $CLASSPATH \
     net.sourceforge.ondex.ovtk2.Main

and then issue ./runme.sh again. You will see that the JVM stops, it has started a debugger server, which is waiting for a connection from a client supporting the Java debugger protocol. This will be Eclipse and the protocol accepts commands like "let's go ahead", "set a breakpoint here", "run one line ahead only". You'll be able to send out these commands from Eclipse, the usual graphical way.

1. In Eclipse, select Run -> Debug Configurations... -> Java Remote Application. Create a new item like 'ONDEX Remote', set knet::launcher as project, set the 5005 connection port. In the source section of this panel, add any projects that you need to debug and for which you need that this configuration is able to see the source code (e.g., if you're debugging a plug-in, add that plug-in there). This is not used to manage dependencies (that is already being done by the JVM you launched at the previous step), but it's used by Eclipse to show you source files when the debugger tells it things like 'I've stopped at this point'.
2. Click on the Run button. You should see things moving on on the shell window. If you select the 'Debug Perspective' in Eclipse, you should see processes and threads that Eclipse sees running. You can stop them, watch variables and execution stack, single-step the execution, set breakpoints etc, as usual.

Debug from another host/server.

The approach above is very general. You can repeat the described steps with any Java application (and pretty much any IDE), including, for instance, a Tomcat Server. You can even run an application from a computer different than your workstation (where Eclipse usually is). Because often you access external hosts/servers via ssh, the only additional step for such a scenario is that you initiate your ssh connection from your personal computer with a port forwarding option like: -L 5005:localhost:5005.

ONDEX Workflow

Basically, it's the same trick as above. Once you've built KNetBuilder, the command-line workflow is available in ondex-knet-builder/ondex-mini/target/*.zip, the .zip is unflattened to ondex-mini/.

ONDEX Integrator

TODO: I don't know how to run the integrator alone, outside of ONDEX. Once we know this, it's the same trick as above.

Maven JUnit tests

This is still based on the Java debugger protocol, the difference here is that Maven Surefire has its own parameters to enable the debugger:

mvn test -Dtest='MyClass#myMethod' -Dmaven.surefire.debug

The the -Dtest parameter is Surefire syntax, it is able to understand that you want to run the tests in a single class (name the class without any '#' trail), or a single method in this class (with the '#', if the method name is unique, the class name can be omitted, alternatively you can pass the FQN of the class, beware that sometimes this doesn't work with certain ONDEX modules, we still don't know why). The debug parameter will start the debugger listener on the 5005 default port. Setting up the Eclipse side works as above.

Starting/Debugging ONDEX from Eclipse

Eclipse has Run/Debug configurations for doing that. A configuration basically tells Eclipse which project you start from, which other dependencies you need to see during your debugging, how the JVM should be started (with the Java main() method, from a JUnit test, starting a web application under Tomcat, etc).

So, in the case of ONDEX, go to the project knet::ovtk2, find and right-click on the class net.sourceforge.ondex.ovtk2.Main and then click 'Debug As...'. Here, give a significant title to the configuration (e.g., ONDEX), and, in the arguments section, set something like this as VM arguments:

-Xmx12G
-Dondex.dir=/home/exuser/ondex-knet-builder/installer/target/ondex/data 
-Dovtk.dir=/home/exuser/ondex-knet-builder/installer/target/ondex/config 
-Dplugin.scan.lib=false

That is, tell ONDEX where the data/ directory and config/ directory are, set these as the directories available in the distribution we have mentioned in the previous paragraph (these are the same parameters that are found in runme.sh in the distribution).

Moreover, in the classpath section, click on 'User Entries', click the 'Advanced', create a new library named eg, 'ONDEX Plugins', having all the jars in ondex-knet-builder/installer/target/plugins/ as content. Save and click on the bottom button 'Debug'. Everything should run as usually.

As for the memory parameter above (-Xmx12G), give ONDEX as much as memory as possible, 2/3 of your total RAM should be enough.

The problem with this method is that you're linking the Eclipse configuration to already-made .jars. If you change any of their code from Eclipse, changes are not reflected immediately in the .jars, you have to first rebuild and replace them for that. Alternatively, you can go back to the Debug Configuration window, go back to the classpath session, add the project (Maven module) you want to debug and place it before the plug-ins library you configured at the previous step. Also, remove the corresponding .jar from that library (just in case, Java should give priority to the project anyway).

Integrator

Works the same way. Define a debug configuration starting from the project knet::ondex-integrator and the class net.sourceforge.ondex.OndexLauncherAll. You also need to set up the parameters:

• Program arguments:

--plugins
/home/exuser/ondex-knet-builder/installer/target/plugins
--data
/home/exuser/ondex-knet-builder/installer/target/ondex/data

(have a look to the main's class for a list of the parameters accepted by the Integrator).

• VM arguments: something like -Xmx12G (as above)

You shouldn't need options in the classpath section. The same reasoning we made above about plug-ins applies here. If you want to debug your own plug-in, add it to the classpath as a project and remove the corresponding .jar from plugins/

Debug a JUnit test

Similarly to the above cases, right-click on the test class or test method and issue Debug As -> JUnit Test. A new debug configuration will be created, possibly adjust its parameters as explained above.

Note: To run Ondex Integrator via Netbeans IDE:

• In Netbeans, under Ondex + Integrator/Workflow tool module -> Run -> VM Options, set: -Xmx6G -Dondex.dir="D:\Program Files (x86)\Ondex\data" (same as in the past, but could also use ondex.dir in installer/target/)
• under Ondex + Integrator/Workflow tool module -> Run -> Arguments, add: --plugins "D:\GitHub_repos\ondex-knet-builder\installer\target\plugins" (to use plugins built at runtime within repo)