configuration specified the fields ["insecure_skip_verify"], but they weren't used in opensearch output plugin

[RodrigoDornelles]

Relevant telegraf.conf

[[outputs.opensearch]]
  template_name = "telegraf"
  urls = ["https://elasticsearch:9200"]
  index_name = "secure-index"
  health_check_interval = "30s"
  username = "admin"
  password = "admin"
  insecure_skip_verify = true

Logs from Telegraf

2023-11-22T18:49:42Z I! Loading config: /etc/telegraf/telegraf.conf
2023-11-22T18:49:42Z E! error loading config file /etc/telegraf/telegraf.conf: plugin outputs.opensearch: line 9: configuration specified the fields ["insecure_skip_verify"], but they weren't used

System info

Telegraf 1.29.0-4c4b821a (git: master@4c4b821a)

Docker

version: '3'

services:  
  elasticsearch:
    image: opensearchproject/opensearch:2.11.0
    environment:
      - cluster.name=my-cluster
      - discovery.type=single-node
      - action.auto_create_index=true
    ports:
      - "9200:9200"
    volumes:
      - ./fake-1/elasticsearch-data:/usr/share/opensearch/data

  telegraf:
    image: telegraf:1.28
    user: "0:0"
    entrypoint: telegraf
    volumes:
      - ./bin/telegraf:/usr/bin/telegraf
      - ./fake-2/telegraf.conf:/etc/telegraf/telegraf.conf:ro
    depends_on:
      - elasticsearch

Steps to reproduce

1. compile the main branch version of telegraf (1.29.0)
2. replace binary of actual telegraf version
3. try to use opensearch like a dev enverioment (without TLS valid)

Expected behavior

Indexed to OpenSearch with status- [201] Result- created DocumentID- 9dhF-IsB-OS_4HM8WCRP 
2023-11-22T18:22:56Z D! [outputs.opensearch] Successfully indexed [1] documents
2023-11-22T18:22:56Z D! [outputs.opensearch] Wrote batch of 1 metrics in 154.781128ms
2023-11-22T18:22:56Z D! [outputs.opensearch] Buffer fullness: 0 / 10000 metrics
indexName- secure-index

Actual behavior

working with bypass

its possible to use in dev enverioment skiping the TLS via reverse proxy.

use nginx to remove TLS

• nginx.conf

events {}

http {
    server {
        listen 9201;
        server_name _;

        location / {
            proxy_pass https://elasticsearch:9200;
            proxy_ssl_verify off;
            proxy_set_body $request_body;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }
}

• docker-compose.yml

version: '3'

services:  
  elasticsearch:
    image: opensearchproject/opensearch:2.11.0
    environment:
      - cluster.name=my-cluster
      - discovery.type=single-node
      - action.auto_create_index=true
    ports:
      - "9200:9200"
    volumes:
      - ./fake-1/elasticsearch-data:/usr/share/opensearch/data

  telegraf:
    image: telegraf:1.28
    user: "0:0"
    entrypoint: telegraf
    volumes:
      - ./bin/telegraf:/usr/bin/telegraf
      - ./fake-2/telegraf.conf:/etc/telegraf/telegraf.conf:ro
    depends_on:
      - elasticsearch
 
  nginx:
    image: nginx:latest
    ports:
      - "9201:9201"
    volumes:
      - ./fake-2/nginx.conf:/etc/nginx/nginx.conf
    depends_on:
      - elasticsearch

• telegraf.conf

[[outputs.opensearch]]
  template_name = "telegraf"
  urls = ["http://nginx:9201"]
  index_name = "insecure-index"
  health_check_interval = "30s"
  username = "admin"
  password = "admin"

output logs

Indexed to OpenSearch with status- [201] Result- created DocumentID- 9dhF-IsB-OS_4HM8WCRP 
2023-11-22T18:22:56Z D! [outputs.opensearch] Successfully indexed [1] documents
2023-11-22T18:22:56Z D! [outputs.opensearch] Wrote batch of 1 metrics in 154.781128ms
2023-11-22T18:22:56Z D! [outputs.opensearch] Buffer fullness: 0 / 10000 metrics
2023-11-22T18:23:05Z D! [outputs.opensearch] indexName- insecure-index

behavior

saved success!

not work and not handle error when TLS is invalid

if you ommit insecure_skip_verify= true does not cause error, but does not save the register.

• telegraf.conf

[[outputs.opensearch]]
  template_name = "telegraf"
  urls = ["https://elasticsearch:9201"]
  index_name = "secure-index"
  health_check_interval = "30s"
  username = "admin"
  password = "admin"

output logs

2023-11-22T18:22:45Z D! [outputs.opensearch] Successfully indexed [1] documents
2023-11-22T18:22:45Z D! [outputs.opensearch] Wrote batch of 1 metrics in 4.424868ms
2023-11-22T18:22:45Z D! [outputs.opensearch] Buffer fullness: 0 / 10000 metrics
2023-11-22T18:22:55Z D! [outputs.opensearch] indexName- secure-index

behavior

failed save!

missing some line like Indexed to OpenSearch with status- [201] Result- created DocumentID- 9dhF-IsB-OS_4HM8WCRP

[srebhan]

@RodrigoDornelles can you please test the binary in PR #14340 available as soon as CI finished the tests!? Let me know if this fixes the issue!

[RodrigoDornelles]

@RodrigoDornelles can you please test the binary in PR #14340 available as soon as CI finished the tests!? Let me know if this fixes the issue!

Of course, I'll do this on Monday because it's on my computer at work or in my environment, and today I'm away!

[RodrigoDornelles]

@srebhan @powersj @Hipska

the PR #14340 solves the problem!