Documentation for stand-alone mode & plugin diagnostics #1719

katarzyna-z · 2017-08-22T11:56:16Z

Summary of changes:

Documentation for standalone plugin
Documentation for plugin diagnostics

Please check if I covered all information related to standalone plugin and plugin diagnostics from user perspective.
I'm open for your suggestion related to this docs 🐢

@intelsdi-x/snap-maintainers

kdembler · 2017-08-22T12:16:32Z

docs/STANDALONE_PLUGIN.md

@@ -0,0 +1,30 @@
+# Standalone Plugin


I think we could go with Standalone Mode instead of Standalone Plugin, as every plugin can be ran in standalone, and Standalone Plugin kinda indicates that it's some special kind of plugin.

kdembler · 2017-08-22T12:17:24Z

docs/STANDALONE_PLUGIN.md

+
+Standalone plugin is a plugin which can be launched on different machine than Snap daemon (`snapteld`).
+
+To run plugin in standalone mode it is needed to launched plugin with `--stand-alone` flag, e.g.:


to launch instead to launched

kdembler

LGTM

IzabellaRaulin · 2017-08-23T07:56:14Z

docs/PLUGIN_DIAGNOSTICS.md

@@ -0,0 +1,114 @@
+# Plugin Diagnostics
+
+Plugin diagnostics provides a simple way to verify that a plugin is capable of running without requiring the plugin to be loaded and a task started.


@katarzyna-z, could we simplify this sentence to be more clear - I will think about some proposal of that. Also, plugin-diagnostics is only collector plugin feature - that should be mentioned somewhere.

IzabellaRaulin · 2017-08-23T07:57:31Z

docs/PLUGIN_DIAGNOSTICS.md

+
+Plugin diagnostics provides a simple way to verify that a plugin is capable of running without requiring the plugin to be loaded and a task started.
+
+It is possible to run plugin diagnostics without any arguments, e.g.:


Add here notice that if the plugin has some configuration needed, the defaults will be taken.

IzabellaRaulin · 2017-08-23T08:29:38Z

docs/PLUGIN_DIAGNOSTICS.md

+```
+
+Plugin diagnostics delivers following information:
+- Runtime details, e.g.:


@katarzyna-z, what do you think about listing all information one by one, and below provide just exemplary output from plugin diagnostics?

Plugin diagnostics delivers following information:

runtime details (plugin name, version, etc.)

configuration details

list of metrics expose by the plugin

metrics with their values that can be collected right now

and times required to retrieve this information

Example:

$ ./snap-plugin-collector-psutil Runtime Details: PluginName: psutil, Version: 14 RPC Type: gRPC, RPC Version: 1 Operating system: linux Architecture: amd64 Go version: go1.7 printRuntimeDetails took 11.878µs Config Policy: NAMESPACE KEY TYPE REQUIRED DEFAULT MINIMUM MAXIMUM intel.psutil.disk mount_points string false printConfigPolicy took 67.654µs Metric catalog will be updated to include: Namespace: /intel/psutil/load/load1 Namespace: /intel/psutil/load/load5 Namespace: /intel/psutil/load/load15 Namespace: /intel/psutil/cpu/*/softirq Namespace: /intel/psutil/cpu/cpu-total/softirq Namespace: /intel/psutil/cpu/*/steal Namespace: /intel/psutil/cpu/cpu-total/steal Namespace: /intel/psutil/cpu/*/guest_nice Namespace: /intel/psutil/cpu/cpu-total/guest_nice Namespace: /intel/psutil/cpu/*/user Namespace: /intel/psutil/cpu/cpu-total/user printMetricTypes took 504.483µs Metrics that can be collected right now are: Namespace: /intel/psutil/load/load1 Type: float64 Value: 1.48 Namespace: /intel/psutil/load/load5 Type: float64 Value: 1.68 Namespace: /intel/psutil/load/load15 Type: float64 Value: 1.65 Namespace: /intel/psutil/cpu/cpu0/softirq Type: float64 Value: 20.36 Namespace: /intel/psutil/cpu/cpu1/softirq Type: float64 Value: 13.62 Namespace: /intel/psutil/cpu/cpu2/softirq Type: float64 Value: 9.96 Namespace: /intel/psutil/cpu/cpu3/softirq Type: float64 Value: 3.6 Namespace: /intel/psutil/cpu/cpu4/softirq Type: float64 Value: 1.42 Namespace: /intel/psutil/cpu/cpu5/softirq Type: float64 Value: 0.69 Namespace: /intel/psutil/cpu/cpu6/softirq Type: float64 Value: 0.54 Namespace: /intel/psutil/cpu/cpu7/softirq Type: float64 Value: 0.31 Namespace: /intel/psutil/cpu/cpu-total/softirq Type: float64 Value: 50.52 Namespace: /intel/psutil/cpu/cpu0/steal Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu1/steal Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu2/steal Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu3/steal Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu4/steal Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu5/steal Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu6/steal Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu7/steal Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu-total/steal Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu0/guest_nice Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu1/guest_nice Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu2/guest_nice Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu3/guest_nice Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu4/guest_nice Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu5/guest_nice Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu6/guest_nice Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu7/guest_nice Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu-total/guest_nice Type: float64 Value: 0 Namespace: /intel/psutil/cpu/cpu0/user Type: float64 Value: 2650.97 Namespace: /intel/psutil/cpu/cpu1/user Type: float64 Value: 2631.81 Namespace: /intel/psutil/cpu/cpu2/user Type: float64 Value: 2523.45 Namespace: /intel/psutil/cpu/cpu3/user Type: float64 Value: 2500.99 Namespace: /intel/psutil/cpu/cpu4/user Type: float64 Value: 2039.26 Namespace: /intel/psutil/cpu/cpu5/user Type: float64 Value: 2082.44 Namespace: /intel/psutil/cpu/cpu6/user Type: float64 Value: 2054.43 Namespace: /intel/psutil/cpu/cpu7/user Type: float64 Value: 1901.43 Namespace: /intel/psutil/cpu/cpu-total/user Type: float64 Value: 18384.81 printCollectMetrics took 7.470091ms showDiagnostics took 8.076025ms

IzabellaRaulin · 2017-08-23T08:42:47Z

docs/PLUGIN_DIAGNOSTICS.md

+    ```
+## Note
+
+For now plugin diagnostics works only for collector plugins.


Ok, now I find it.

Do we need to specify that diagnostics only works with plugins written using one of the plugin libs.

Also, should we specify that diagnostics works with both collectors and streaming collectors?

@kjlyon Great ideas! It is important that plugin must be written using one of our libs and diagnostics works also for streaming collectors.

IzabellaRaulin · 2017-08-23T08:46:05Z

docs/STANDALONE_MODE.md

+$  snaptel plugin load http://localhost:8182
+```
+
+The rest of operations remains exactly the same as is it for regular plugins.


Using regular plugins might be misleading. Maybe the following is better: The rest of operations remains exactly the same as is it for plugins running in regular mode.

kjlyon

I found a couple places that could be reworded. I'm also thinking we need to add documentation about how to write a plugin that has diagnostics and stand-alone mode enabled. That would involve acknowledging the plugin libs, I don't think it needs to be more than a sentence or two. Thoughts?

kjlyon · 2017-08-23T20:45:31Z

docs/PLUGIN_DIAGNOSTICS.md

+    ```
+## Note
+
+For now plugin diagnostics works only for collector plugins.


Do we need to specify that diagnostics only works with plugins written using one of the plugin libs.

kjlyon · 2017-08-23T20:49:26Z

docs/PLUGIN_DIAGNOSTICS.md

+    ```
+## Note
+
+For now plugin diagnostics works only for collector plugins.


Also, should we specify that diagnostics works with both collectors and streaming collectors?

kjlyon · 2017-08-23T20:57:46Z

docs/PLUGIN_DIAGNOSTICS.md

+        Namespace: /intel/psutil/cpu/cpu6/user     Type: float64     Value: 2054.43 
+        Namespace: /intel/psutil/cpu/cpu7/user     Type: float64     Value: 1901.43 
+        Namespace: /intel/psutil/cpu/cpu-total/user  Type: float64     Value: 18384.81
+       [...]


Could we shorten this list? I think we can say ... much sooner and users will get the idea without having to list all these metrics.

kjlyon · 2017-08-23T21:04:37Z

docs/STANDALONE_MODE.md

@@ -0,0 +1,30 @@
+# Standalone mode


The flag currently is --stand-alone with the hyphen. In this document standalone is used. These should match. For usability I think the flag should change to one word, but for ease, it makes more sense to just change the documentation to match the flag.

I don't think that it is good idea to change now --stand-alone flag to --standalone because some of our plugins have been update to new versions of plugin libs to provide diagnostics information. I think that in documentation I can change "stanalone mode" to "stand-alone mode". @kjlyon what do you think?

In the future if someone decide to change flag he/she should also take into account that it is needed to update all plugins which use old versions of snap-plugin-libs.

kjlyon · 2017-08-23T21:05:41Z

docs/STANDALONE_MODE.md

@@ -0,0 +1,30 @@
+# Standalone mode
+
+Standalone mode enables plugin launching on different machine than Snap daemon (`snapteld`).


Rather than launching, I think start would be a better word. Like this:
Stand-alone mode enables plugins to start and run on a different machine than Snap daemon (`snapteld`).

kjlyon · 2017-08-23T21:07:05Z

docs/STANDALONE_MODE.md

+
+Standalone mode enables plugin launching on different machine than Snap daemon (`snapteld`).
+
+To run plugin in standalone mode it is needed to launch plugin with `--stand-alone` flag, e.g.:


Reword: To run a plugin in stand-alone mode, you must start it with the `--stand-alone` flag, e.g.:

kjlyon · 2017-08-23T21:11:42Z

docs/STANDALONE_MODE.md

+```
+$ ./snap-plugin-collector-psutil --stand-alone
+```
+Plugin in standalone mode creates HTTP server for communication with Snap framework, by default plugin listens on `8182` port.


Reword: A plugin running in stand-alone mode will create a HTTP server for communication with the Snap framework. By default the plugin listens on port `8182`.

kjlyon · 2017-08-23T21:13:31Z

docs/STANDALONE_MODE.md

+```
+Plugin in standalone mode creates HTTP server for communication with Snap framework, by default plugin listens on `8182` port.
+
+It is possible to configure different port for communication with Snap framework using `--stand-alone-port` flag, e.g.:


Reword: To specify a different listening port, use the `--stand-alone-port` flag, e.g.:

kjlyon · 2017-08-23T21:19:49Z

docs/STANDALONE_MODE.md

+$ ./snap-plugin-collector-psutil --stand-alone --stand-alone-port 8183
+```
+
+To load plugin in standalone mode it is required to provide URL to indicate a machine


I think this section should have its own header so that it stands out more.

@kjlyon Do you have any proposition of the header?

Something like Loading a plugin in snapteld or simply, Loading a plugin

kjlyon · 2017-08-23T21:22:08Z

docs/STANDALONE_MODE.md

+$ ./snap-plugin-collector-psutil --stand-alone --stand-alone-port 8183
+```
+
+To load plugin in standalone mode it is required to provide URL to indicate a machine


Possible reword: When loading a stand-alone plugin into snapteld you must provide a URL to idicate which machine the plugin is running (IP address/hostname with port number), e.g.:

@kjlyon thanks!

kjlyon · 2017-08-23T21:33:08Z

I'm not sure if this is the right location for this, but in the documentation somewhere we should mention that if the connection between snapteld and the plugin running in stand-alone mode is disrupted it will try to reconnect. If it is unable to reconnect the plugin will be unloaded, metric catalog updated, and any dependent tasks will stop.

IzabellaRaulin · 2017-08-25T07:14:50Z

According to the last comment:

I'm not sure if this is the right location for this, but in the documentation somewhere we should mention that if the connection between snapteld and the plugin running in stand-alone mode is disrupted it will try to reconnect. If it is unable to reconnect the plugin will be unloaded, metric catalog updated, and any dependent tasks will stop.

I agree. There might be added notice about known issue with brief description and link to the GH issue (1697)

katarzyna-z · 2017-08-25T09:05:47Z

@kjlyon

I'm also thinking we need to add documentation about how to write a plugin that has diagnostics and stand-alone mode enabled.

There is nothing special to implement in plugin to enable diagnostics and standalone mode. Diagnostics and standalone mode work for any plugin which uses one of our libs and in my opinion this information will be enough.

kjlyon · 2017-08-25T14:44:38Z

@katarzyna-z

There is nothing special to implement in plugin to enable diagnostics and standalone mode. Diagnostics and standalone mode work for any plugin which uses one of our libs and in my opinion this information will be enough.

I agree, this information is enough, but that we should add a sentence in each of these documents saying that plugins using our libs will have these features.

katarzyna-z · 2017-08-28T13:46:21Z

@kjlyon @IzabellaRaulin @kdembler I've updated this PR. Please take a look at this once again. Let me know if I forgot about something ;)

kdembler · 2017-08-28T13:58:07Z

docs/PLUGIN_DIAGNOSTICS.md

+Plugin diagnostics delivers following information:
+- runtime details (plugin name, version, etc.)
+- configuration details
+- list of metrics expose by the plugin


Typo in exposed

kdembler · 2017-08-28T13:59:33Z

docs/PLUGIN_DIAGNOSTICS.md

+$ ./snap-plugin-collector-psutil --config {\"mouting_points\":\"/dev/sda\"}
+```
+
+For example:


Maybe this could be changed to Example output:, but it's just a suggestion.

kjlyon

LGTM

IzabellaRaulin · 2017-08-29T08:39:33Z

docs/PLUGIN_DIAGNOSTICS.md

+```
+or with plugin configuration in the form of a JSON:
+```
+$ ./snap-plugin-collector-psutil --config {\"mouting_points\":\"/dev/sda\"}


@katarzyna-z, could you check the way how to provide the config? Ssee my example below, but please verify that. The another thing is that the psutil collector does not have mouting_points config param, but mount_points, so it's better to use config's name which really exists

$ ./snap-plugin-collector-psutil --config '{"mount_points":"/dev/sda"}'

IzabellaRaulin · 2017-08-29T08:45:41Z

docs/STAND-ALONE_MODE.md

+```
+##  Loading a plugin
+When loading a plugin in stand-alone mode into `snapteld` you must provide a URL to indicate which
+machine the plugin is running (IP address/hostname with port number), e.g.:


Consider rephrasing
When loading a plugin in stand-alone mode into snapteld you must provide a URL to indicate which machine the plugin is running (IP address/hostname with port number), e.g.:
to

To load a plugin in stand-alone mode, provide a URL to indicate to the machine on which the plugin is running (IP address/hostname with port number), e.g.:

IzabellaRaulin · 2017-08-29T08:59:32Z

docs/STAND-ALONE_MODE.md

+## Notice
+
+If there is any disruption in the connection between Snap and a stand-alone plugin then the task is disabled and the plugin is unloaded,
+see [github issue](https://github.com/intelsdi-x/snap/issues/1697).


What do you think about renaming the section to "Known issues" instead of "Notice" and rephrasing its content:

If some disruption occurs in the connection between Snap and a stand-alone plugin, the running task will be stopped with disabled status and the plugin will be unloaded. Providing the mechanism of reconnecting stand-alone plugins upon network disruption is in our scope, addressed by issue #1697.

IzabellaRaulin · 2017-08-29T09:50:31Z

@katarzyna-z, I created a PR to your branch with addressed my comments - katarzyna-z#1. Besides these, LGTM

Addressed comments to docs stand alone and diagnostics

kdembler reviewed Aug 22, 2017

View reviewed changes

katarzyna-z force-pushed the docs-stand-alone-diganostics branch 2 times, most recently from b0530ca to 95f00a8 Compare August 22, 2017 12:52

kdembler approved these changes Aug 22, 2017

View reviewed changes

IzabellaRaulin reviewed Aug 23, 2017

View reviewed changes

kjlyon suggested changes Aug 23, 2017

View reviewed changes

katarzyna-z force-pushed the docs-stand-alone-diganostics branch 3 times, most recently from 4fe6e17 to 3dfa51d Compare August 28, 2017 13:42

katarzyna-z force-pushed the docs-stand-alone-diganostics branch 2 times, most recently from 5bca749 to 6b992c4 Compare August 28, 2017 13:55

kdembler reviewed Aug 28, 2017

View reviewed changes

katarzyna-z force-pushed the docs-stand-alone-diganostics branch 2 times, most recently from 75c038f to 1a3a7cd Compare August 28, 2017 14:13

Documentation for stand-alone mode & plugin diagnostics

1a3a7cd

kjlyon approved these changes Aug 28, 2017

View reviewed changes

IzabellaRaulin suggested changes Aug 29, 2017

View reviewed changes

IzabellaRaulin added 2 commits August 29, 2017 11:44

Update PLUGIN_DIAGNOSTICS.md

fa99bb7

Update STAND-ALONE_MODE.md

3504ca7

Merge pull request #1 from IzabellaRaulin/docs-stand-alone-diganostics

2b3f41d

Addressed comments to docs stand alone and diagnostics

IzabellaRaulin approved these changes Aug 29, 2017

View reviewed changes

IzabellaRaulin merged commit 84ef5a9 into intelsdi-x:master Aug 29, 2017

katarzyna-z mentioned this pull request Aug 29, 2017

Add doc for standalone plugin intelsdi-x/snap-plugin-lib-go#94

Closed


		Standalone plugin is a plugin which can be launched on different machine than Snap daemon (`snapteld`).

		To run plugin in standalone mode it is needed to launched plugin with `--stand-alone` flag, e.g.:

		@@ -0,0 +1,114 @@
		# Plugin Diagnostics

		Plugin diagnostics provides a simple way to verify that a plugin is capable of running without requiring the plugin to be loaded and a task started.


		Plugin diagnostics provides a simple way to verify that a plugin is capable of running without requiring the plugin to be loaded and a task started.

		It is possible to run plugin diagnostics without any arguments, e.g.:

		@@ -0,0 +1,30 @@
		# Standalone mode

		Standalone mode enables plugin launching on different machine than Snap daemon (`snapteld`).

Documentation for stand-alone mode & plugin diagnostics #1719

Documentation for stand-alone mode & plugin diagnostics #1719

Conversation

katarzyna-z commented Aug 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kdembler left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

IzabellaRaulin Aug 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kjlyon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kjlyon commented Aug 23, 2017

IzabellaRaulin commented Aug 25, 2017 • edited Loading

katarzyna-z commented Aug 25, 2017

kjlyon commented Aug 25, 2017 • edited Loading

katarzyna-z commented Aug 28, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kjlyon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

IzabellaRaulin commented Aug 29, 2017

IzabellaRaulin Aug 23, 2017 •

edited

Loading

IzabellaRaulin commented Aug 25, 2017 •

edited

Loading

kjlyon commented Aug 25, 2017 •

edited

Loading