x509 Certificate Input Plugin v1.19.0 does not work with URL

[aslgithub]

Relevant telegraf.conf:

[[inputs.x509_cert]]
sources = ["https://my.server.com:443"] 

System info:

Windows Server 2019
Telegraf v1.19.0

Steps to reproduce:

1. Configure inputs.x509_cert
2. telegraf.exe --config "C:\Program Files\Telegraf\telegraf.conf" --test --input-filter x509_cert --debug

Expected behavior:

Certificate details returned

x509_cert,common_name=*.server.com,country=GB
x509_cert,common_name=Root Cert

Actual behavior:

No certificate details are returned
2021-06-16T16:52:07Z E! [inputs.x509_cert] could not find file: &{https:\my.server.com:443 false false }

Additional info:

Telegraf v1.18.1 on the same system does not show this issue.

[aslgithub]

Breaking change seems to be #6952 which has broken x509_cert https:// and other prefixes

[pradig]

Same problem under Linux, too.

[srebhan]

@aslgithub and @pradig can you please test PR #9289? This should fix the problem... Would be good to get some confirmation that the issue is gone. The Windows artifact can be found here, trying to get the linux one built...

[srebhan]

Ok all artifacts built by CI can be found here including Linux. Just click on that small black rectangle to unfold the list.

Would appreciate your feedback...

[pradig]

Works for me now, Ubuntu 20.04.2
Thanks!

[aslgithub]

@srebhan
Its now providing an output rather than an error; but the behaviour does not match pre #6952

[[inputs.x509_cert]]
sources = ["https://my.server.com:443"]

v1.18.1 output snip
x509_cert,source=https://my.server.com:443

This artifact build
source=tcp://my.server.com:443

notice the https is now assumed to be tcp

If I remove the port in my config
2021-06-18T09:57:25Z E! [inputs.x509_cert] Error in plugin: cannot get SSL cert 'tcp://my.server.com': dial tcp: address my.server.com: missing port in address

So it looks as if its now assuming everything is tcp ?
Sorry !

[pradig]

@aslgithub

These are my entries, and they work perfectly well:

"https://www.xxx.com:443",
"https://minio.yyy.net:9000",

[aslgithub]

@pradig
and your actual output within the x509_cert contains source="https://www.xxx.com:443"
rather than source="tcp://www.xxx.com:443" ?

In which case this is a Windows versus Ubuntu issue..

[pradig]

This is the output:

x509_cert,common_name=www.xxx.com,host=orfeo,issuer_common_name=R3,public_key_algorithm=RSA,san=www.xxx.com,serial_number=3a220f4f2b064ad3a4c0d84436cbbe0a1ae,signature_algorithm=SHA256-RSA,source=tcp://www.xxx.com:443,verification=valid age=3551030i,enddate=1628236328i,expiry=4224969i,startdate=1620460328i,verification_code=0i 1624011359000000000

[aslgithub]

okay, so you have the same issue
you are requesting :
"https://www.xxx.com:443"
but the output back is
source=tcp://www.xxx.com:443

If you compare this to v1.18.0 for instance
"https://www.xxx.com:443"
but the output back is
source=https://www.xxx.com:443

Thanks,
Paul

[pradig]

I got your point now...
before telegraf 1.18 it was source=https://www.xxx.com:443
and with telegraf 1.19 it is now source=tcp://www.xxx.com:443
So we still have a little problem here.

[pradig]

I just had a look at the x509_cert documentation. It is a little bit inconsistent, the example config uses "tcp://example.org:443" but the sample output has "source=https://example.org:443".
So this is the reverse behaviour from what we are seeing now with 1.19, and it is still crazy for me. I would expect "tcp://" in and "tcp:://" out, or "https://" in and "https://" out.

[aslgithub]

#6952 seems to be 'bad' that made these changes.

If you look at
https://github.com/influxdata/telegraf/pull/6952/files

It updated the readme to remove https from the example

## List certificate sources sources = ["/etc/ssl/certs/ssl-cert-snakeoil.pem", "https://example.org:443"]

And changed it to

## List certificate sources, support wildcard expands for files ## Prefix your entry with 'file://' if you intend to use relative paths sources = ["/etc/ssl/certs/ssl-cert-snakeoil.pem", "tcp://example.org:443", "/etc/mycerts/*.mydomain.org.pem", "file:///path/to/*.pem"]

But didn't update the example output which still shows as https correctly.

I'd expect doing certificate checks against https is fairly fundamental, at least in our use, and how this is being requested as https through firewalls and proxies, and the source is displayed/selected/filtered.

[pradig]

Well, I reckon that "tcp:" would be technically ok (as certificates are part of TLS, and this is agnostic of any higher level protocol, like https) -- but it is not what you would expect if you checked the certificate of a classical web site (which I guess is the major use case of this plugin).

[srebhan]

Thanks for testing folks! I think I found the issue and prepared a fix, also comprising the changes in #9289 for testing. Can you give the artifacts in #9400 a try as soon as they are built?
Thanks a lot and sorry for the inconvenience this whole mess might have caused.

[pradig]

Confirmed, fixes it for me.
Thanks for this fix at light speed.

[aslgithub]

@srebhan also confirmed as fixing for me, thank you. The request now matches the source.
Just a note to say I really appreciate your efforts Sven, you got the SQL input over the line into v1.19, but I'm now stuck on this, the SQL integrated login issue, and RabbitMQ aarrgghhh so close !

[srebhan]

@pradig thanks for testing!
@aslgithub thanks for testing! Please keep this on-topic! ;-) You can contact me on slack or discuss the matter on the related ticket (if there is one already).