This plugin for the Cloud Foundry Command Line provides convenience utilities to work with Java applications deployed on Cloud Foundry.
Currently, it allows to:
- Trigger and retrieve a heap dump from an instance of a Cloud Foundry Java application
- Trigger and retrieve a thread dump from an instance of a Cloud Foundry Java application
Make sure you have the CF Community plugin repository configured or add it via (cf add-plugin-repo CF-Community http://plugins.cloudfoundry.org
)
Trigger installation of the plugin via
cf install-plugin -r CF-Community "java-plugin"
Download the binary file for your target OS from the latest release.
If you've already installed the plugin and are updating it, you must first execute the cf uninstall-plugin JavaPlugin
command.
Install the plugin with cf install-plugin [cf-cli-java-plugin]
(replace [cf-cli-java-plugin]
with the actual binary name you will use, which depends on the OS you are running).
You can verify that the plugin is successfully installed by looking for JavaPlugin
in the output of cf plugins
.
On Linux and macOS, if you get a permission error, run chmod +x [cf-cli-java-plugin]
(replace [cf-cli-java-plugin]
with the actual binary name you will use, which depends on the OS you are running) on the plugin binary.
On Windows, the plugin will refuse to install unless the binary has the .exe
file extension.
NAME: java - Obtain a heap dump or thread dump from a running, Diego-enabled, SSH-enabled Java application USAGE: cf java [heap-dump|thread-dump] APP_NAME OPTIONS: -app-instance-index -i [index], select to which instance of the app to connect -dry-run -n, just output to command line what would be executed -keep -k, keep the heap dump in the container; by default the heap dump will be deleted from the container's filesystem after been downloaded
The heap dump or thread dump (depending on what you execute) will be outputted to std-out
.
You may want to redirect the command's output to file, e.g., by executing:
cf java heap-dump [my_app] -i [my_instance_index] > heap-dump.hprof
The -k
flag is invalid when invoking cf java thread-dump
.
(Unlike with heap dumps, the JVM does not need to output the threaddump to file before streaming it out.)
As it is built directly on cf ssh
, the cf java
plugin can work only with Diego applications that have cf ssh
enabled.
To check if your app fulfills the requirements, you can find out by running the cf ssh-enabled [app-name]
command.
Also, cf ssh
is very picky with proxies.
If cf java
is having issues connecting to your app, chances are the problem is in the networking issues encountered by cf ssh
.
To verify, run your cf java
command in "dry-run" mode by adding the -n
flag and try to execute the command line that cf java
gives you back.
If it fails, the issue is not in cf java
, but in whatever makes cf ssh
fail.
The capability of creating heap dumps is also limited by the filesystem available to the container.
The cf java heap-dump
command triggers the heap dump to file system, read the content of the file over the SSH connection, and then remove the heap dump file from the container's file system (unless you have the -k
flag set).
The amount of filesystem space available to a container is set for the entire Cloud Foundry landscape with a global configuration.
The size of a heap dump is roughly linear with the allocated memory of the heap.
So, it could be that, in case of large heaps or the filesystem having too much stuff in it, there is not enough space on the filesystem for creating the heap dump.
In that case, the creation of the heap dump and thus the command will fail.
From the perspective of integration in workflows and overall shell-friendliness, the cf java
plugin suffers from some shortcomings in the current cf-cli
plugin framework:
- There is no distinction between
stdout
andstderr
output from the underlyingcf ssh
command (see this issue on thecf-cli
project)- The
cf java
will however exit with status code1
when the underpinningcf ssh
command fails - If split between
stdout
andstderr
is needed, you can run thecf java
plugin in dry-run mode (--dry-run
flag) and execute its output instead
- The
- The plugin is not current capability of storing output directly to file (see this issue on the
cf-cli
project)- The upstream change needed to fix this issue has been scheduled at Pivotal; when they provide the new API we need, we'll update the
cf java
command to save output to file.
- The upstream change needed to fix this issue has been scheduled at Pivotal; when they provide the new API we need, we'll update the
Executing a thread dump via the cf java
command does not have much of an overhead on the affected JVM.
(Unless you have a lot of threads, that is.)
Heap dumps, on the other hand, have to be treated with a little more care. First of all, triggering the heap dump of a JVM makes the latter execute in most cases a full garbage collection, which will cause your JVM to become unresponsive for the duration. How much time is needed to execute the heap dump, depends on the size of the heap (the bigger, the slower), the algorithm used and, above all, whether your container is swapping memory to disk or not (swap is bad for the JVM). Since Cloud Foundry allows for over-commit in its cells, it is possible that a container would begin swapping when executing a full garbage collection. (To be fair, it could be swapping even before the garbage collection begins, but let's not knit-pick here.) So, it is theoretically possible that execuing a heap dump on a JVM in poor status of health will make it go even worse.
Secondly, as the JVMs output heap dumps to the filesystem, creating a heap dump may lead to to not enough space on the filesystem been available for other tasks (e.g., temp files). In that case, the application in the container may suffer unexpected errors.
The tests are written using Ginkgo with Gomega for the BDD structure, and Counterfeiter for the mocking generation.
Unless modifications to the helper interfaces cmd.CommandExecutor
and uuid.UUIDGenerator
are needed, there should be no need to regenerate the mocks.
To run the tests, go to the root of the repository and simply run gingko
(you may need to install Ginkgo first, e.g., go get github.com/onsi/ginkgo/ginkgo
puts the executable under $GOPATH/bin
).