How to create the 'dockercontainer' sensor?

[H0wi]

Hey.

I'm trying to add the sensor for dockercontainers via the Powershell CLI

My code so far:

$Device = Get-Device -Id 1754

$Key = "-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----"

$Cert = "-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----"

$Params = $Device | New-SensorParameters -Verbose -RawType "dockercontainer" -QueryParameters @{
    dockerport_ = 2376
    key_ = "$Key"
    cert_ = "$Cert"
}
$Params

And I get the following error
New-SensorParameters : An unspecified error occurred while trying to resolve sensor targets. Specified sensor type may not be valid on this device, or sensor query target parameters may be incorrect. Check the Device 'Host' is still valid or try adding sensor with the PRTG UI.

I tried some variations and debugging the script and the creation process via the PRTG UI and noticed that the UI sends a POST request to addsensor3.htm but with my code a GET request will be sent.

Does anyone have any idea what I'm doing wrong or has a hint?
I feel like I'm missing something.

Thanks for your Help.

[lordmilko]

I tested your code on my own Docker instance and it worked fine. Are you sure the ID your Docker server's PRTG device is 1754? I only get the error you get when I target a server that is not running the Docker Daemon

My Docker Daemon runs on Windows; the contents of my daemon.json are as follows

{
    "hosts": ["tcp://0.0.0.0:2376", "npipe://"],
    "data-root": "d:\\docker",
    "experimental":  true,
    "tlsverify": false,
    "tlscacert": "D:\\docker\\certs.d\\ca.pem",
    "tlscert": "D:\\docker\\certs.d\\server-cert.pem",
    "tlskey": "D:\\docker\\certs.d\\server-key.pem"
}

[H0wi]

Well, the device ID is not 1754 because I just entered a random number in the code excerpt.
But I'm sure I used the right id.

I think I had some encoding issues with the key and cert string but this should be fixed now. I guess this was the reason the connection to the docker-daemon failed. I also noticed that I haven't used the most recent version. (I used 0.9.7)

Now I tried to send the requests (addsensor2, addsensor3 and getaddsensorprogress from the debug output) via Postman and it worked.

But now I'm getting the following Error after the addsensor3 request was sent:

New-SensorParameters : Failed to resolve sensor targets for sensor type 'dockercontainer': type was not valid or you do not have sufficient permissions on the specified object.

Adding other sensors is still possible. I have tried it with other hosts/device IDs as well and it did not work.

The code I'm using now:

if (-not (Get-PRTGClient)) {
  Connect-PrtgServer -Server $PRTG_SERVER
}

$Key = '-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----'

$Cert = '-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----'

$Device = Get-Device -Id 2690 -Verbose
$Device

# Works fine.
$Params = $Device | New-SensorParameters -RawType "ping"
$Params

# Does not work :(
$Params = $Device | New-SensorParameters -Verbose -RawType 'dockercontainer' -QueryParameters @{
    dockerport_ = 2376
    key_ = "$Key"
    cert_ = "$Cert"
}
$Params

The debug output:

[...]
VERBOSE: New-SensorParameters: Synchronously executing request https://<PRTGServer>/api/sensortypes.json?id=2690&username=XXX&passhash=XXX
VERBOSE: New-SensorParameters: Synchronously executing request https://<PRTGServer>/controls/addsensor2.htm?id=2690&sensortype=dockercontainer&username=XXX&passhash=XXX
VERBOSE: New-SensorParameters: Synchronously executing request https://<PRTGServer>/controls/addsensor3.htm?id=2690&tmpid=519&dockerport_=2376&key_=-----BEGIN+PRIVATE+KEY-----[..]-----END+PRIVATE+KEY-----&cert_=-----BEGIN+CERTIFICATE-----[..]-----END+CERTIFICATE-----
New-SensorParameters : Failed to resolve sensor targets for sensor type 'dockercontainer': type was not valid or you do not have sufficient permissions on the specified object.

I really appreciate your help.

[lordmilko]

Are you potentially able to elaborate on the "encoding issues" you're referring to? In my test I simply copied and pasted the details from my *.pem files; I'm not sure if your certificates include some random characters that potentially caused issues?

PrtgAPI uses the following flow to implement the logic required by this API

1. ResolveSensorTargets calls GetSensorTargetsResponse
2. GetSensorTargetsResponse calls GetAddSensorTmpId
3. GetAddSensorTmpId first executes a request against addsensor2.htm, and then passes the response of that request to GetSensorTargetTmpId
4. GetSensorTargetTmpId executes a request addsensor3.htm at the very end of the method, containing all of the -QueryParameters values and attempts to extract the tmpid returned from the addsensor3.htm request to use as the request response
5. GetSensorTargetsResponse validates that a tmpid was actually returned. If there wasn't, implicitly PRTG ran into some sort of issue and the exception you're seeing is thrown

When you execute requests using the PRTG UI, generally speaking PRTG uses POST in conjunction with cookies and implements appropriate systems to always keep your cookies "active". Based on my testing, using POST in conjunction with the standard username/passhash fields does not work. As evidence has shown we can get away with using GET for almost all requests, we use GET almost everywhere in PrtgAPI, and haven't needed to develop our own cookie management system.

As it happens addsensor3.htm is one of the few API requests that does require the use of cookies; that's fine however, as we can simply store some cookies from the requests that came before it and then inject the cookie into the request, instead of the username and passhash. In this way we don't have to worry about cookies being expired, which we'd have to deal with if PrtgAPI almost exclusively used cookies

Some questions:

• What version of PRTG are you running?
• If you run Fiddler are you able to see anything suspicious about the request that is executed to addsensor3.htm? What HTTP status code is returned? What is the response URI? (most important)
• Did you use GET or POST in Postman? Did you have to do anything funny involving cookies to make it work? If you used POST, does it work if you instead try with GET (along with potentially any other required adjustments)?
• In my testing previously, if you mess with your certificate values you'll get an error telling you that they are invalid. Potentially it could be worth investigating what happens when you try entering random values like "potato" I can't reproduce this anymore so nevermind

[lordmilko]

Also interesting is that my certificate key begins with BEGIN RSA PRIVATE KEY, whereas yours is just BEGIN PRIVATE KEY. I wonder if regenerating your certificates to be the same format as mine would help?

These are the commands I used to generate the required *.pem files from my domain's wildcard SSL PFX

openssl pkcs12 -in "wildcard.pfx" -nocerts -out privatekey.pem -nodes
openssl pkcs12 -in "wildcard.pfx" -nokeys -out publiccert.pem -nodes
openssl pkcs12 -in "wildcard.pfx" -cacerts -nokeys -out ca.pem -nodes

(obviously you'll want to backup your existing certs and adjust the filenames/daemon.json paths appropriately)

[H0wi]

Okay, so first I tried to get the certificate and the key directly from the files via $Key = Get-Content prtg.key and $Cert = Get-Content prtg.crt but I noticed that the encoding in the addsensor3.htm call was different than writing it directly as a string in the file. Also I noticed that the carriage return/line feed in the ps1-script had an impact on the request send (I'm using CRLF). I'm not sure anymore if that was really the problem, because yesterday I tried it with the certificate in the scipt as well ...

I'm using PRTG Version 19.2.50.2842

I used GET requests only in Postman without touching the cookie tab (maybe postman does something automatically?).
I executed the addsensor2.htm (copied it from the script-log), addsensor3.htm (with the URL-encoded cert from the script-log and replaced the tmpid with the returned one), getaddsensorprogress.htm and then addsensor4.htm (which then containts a table with the running containers on the host).

My results from trying to debugg the HTTPS traffic to the PRTG-server via Fiddler:

GET /api/sensortypes.json?id=2690&username=XXX&passhash=XXX
GET /controls/addsensor2.htm?id=2690&sensortype=dockercontainer&username=XXX&passhash=XXX
GET /controls/addsensor3.htm?id=2690&tmpid=527&dockerport_=2376&key_=-----BEGIN+PRIVATE+KEY-----%0D%0AMIIJQgIBADANBgkqhkiG9w0BAQE[...]0AabAY0kD65%2Bnsn66rgZpXvcZ5bm5U0g%3D%3D%0D%0A-----END+PRIVATE+KEY-----&cert_=-----BEGIN+CERTIFICATE-----%0D%0AMIIE1zCC[...]%3D%0D%0A-----END+CERTIFICATE-----

All there requests returned a HTTP 200 status

The last request returned:

<center>
	<p>Preparing sensor settings...</p>
	<div id="newprogressbar"></div>

	<input type="hidden" id="tempid" name="tempid" value="537">
		  </center>

A request to getaddsensorprogress.htm was not sent.

I then copied the requests from Fiddler and tried sending the requests via Postman. I got the same result from the addsensor3.htm request (Preparing sensor settings...) and then executed the following requests:

GET /api/getaddsensorprogress.htm?id=2855&tmpid=538:

{"progress":100,"targeturl":"/addsensor4.htm?id=2855&tmpid=538"}

GET /addsensor4.htm?id=2855&tmpid=538:

[...]
<div class="controls fillwidth" data-caption="Container">
  <table class="table hoverable datatables container_ ">
    <thead>
      <tr>
        <th class="no_sort checkboxholder">
          <input type="checkbox" class="checkbox checkboxtoggler" id="checkboxtogglercontainer_"><label
            for="checkboxtogglercontainer_"></label><input type="hidden" name="container_" value="1"></th>
        <th>Name</th>
        <th>Image</th>
        <th>Status</th>
      </tr>
    </thead>
    <tr class=" notconnected  even">
      ...
    </tr>
    <tr class=" notconnected  odd">
      ...
    </tr>
    <tr class=" notconnected  even">
      ...
    </tr>
    <tr class=" notconnected  odd">
      ...
    </tr>
  </table>
</div>
[...]

which worked just fine and returned a table with the containers running on the host.

[lordmilko]

I think I may have solved it!

Are you doing this in PowerShell or the PowerShell ISE? If PowerShell, can you try using the ISE instead?

I noticed when I type everything up in the ISE and hit F5 it runs without issue. When I save that script, open a regular PowerShell and then run the script, I get the same error as you! How can the same script generate different results in different circumstances?

The answer is that the response URI returned in PowerShell contains newlines (\n) whereas in the ISE the URI is all a single line. This appears to trip up our attempt to extract the tmpid from the response, resulting in a tmpid of <number>\n<certificate>. PrtgAPI later tries to convert this string to a number, fails, and assumes it didn't get a tmpid.

The solution to this is to simply modify the tmpid extraction regex to treat the input string as a single line. I'm not sure whether you need to be running your code in a script automatically all the time or whether this is just a one time thing, however are you able to advise if using the ISE resolves the issue?

Regards,
lordmilko

[H0wi]

Yesss, via the ISE it works :)

The plan was to start it automatically. What do I have to do for it? I didn't quite understand that part.

Thank you for your efforts.

[lordmilko]

Well, I haven't figured that out yet :P Every workaround I've tried so far doesn't work; the only way around it appears to be to fix it properly, internally.

I'm not yet ready to release the next version of PrtgAPI, however in the meantime you can potentially clone the repo, apply the fix and then recompile it yourself

1. Open a Command Prompt Run git clone https://github.com/lordmilko/PrtgAPI
2. Change this line from

internal static PrtgResponse GetSensorTargetTmpId(HttpResponseMessage message) => Regex.Replace(message.RequestMessage.RequestUri.ToString(), "(.+tmpid=)(\\d+)(.*)", "$2");

to

internal static PrtgResponse GetSensorTargetTmpId(HttpResponseMessage message) => Regex.Replace(message.RequestMessage.RequestUri.ToString(), "(.+tmpid=)(\\d+)(.*)", "$2", RegexOptions.Singleline);

3. Open build.cmd in the root of the repo and run Invoke-PrtgBuild -Configuration Release. Any required dependencies should be automatically downloaded; if all goes well, you'll have an updated version of PrtgAPI under PrtgAPI\src\PrtgAPI.PowerShell\bin\Release\net452\PrtgAPI

[lordmilko]

Hi @H0wi,

Please be advised PrtgAPI 0.9.10 has been released, which includes a fix for this issue. To update to the latest version of PrtgAPI run the command

Update-Module PrtgAPI

and then close and reopen PowerShell.

Regards,
lordmilko