[AUTOCUT] Integration Test failed for sql: 2.9.0 tar distribution

[opensearch-ci-bot]

The integration test failed at distribution level for component sql
Version: 2.9.0
Distribution: tar
Architecture: x64
Platform: linux

Please check the logs: https://build.ci.opensearch.org/job/integ-test/5321/display/redirect

* Steps to reproduce: See https://github.com/opensearch-project/opensearch-build/tree/main/src/test_workflow#integration-tests
* Access components yml file:
- With security (if applicable)
- Without security (if applicable)

Note: All in one test report manifest with all the details coming soon. See opensearch-project/opensearch-build#1274

[Yury-Fridlyand]

Suite: Test class org.opensearch.sql.datasource.DataSourceAPIsIT
  2> REPRODUCE WITH: ./gradlew ':integ-test:integTest' --tests "org.opensearch.sql.datasource.DataSourceAPIsIT.createDataSourceAPITest" -Dtests.seed=CF0D2A58F0EDD5EC -Dtests.security.manager=false -Dtests.locale=sk-SK -Dtests.timezone=MET -Druntime.java=17
  2> org.opensearch.client.ResponseException: method [POST], host [http://localhost:9200/], URI [/_plugins/_query/_datasources], status line [HTTP/1.1 400 Bad Request]
    {
      "status": 400,
      "error": {
        "type": "IllegalStateException",
        "reason": "Invalid Request",
        "details": "Master key is a required config for using create and update datasource APIs.Please set plugins.query.datasources.encryption.masterkey config in opensearch.yml in all the cluster nodes. More details can be found here: https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/admin/datasources.rst#master-key-config-for-encrypting-credential-information"
      }
    }
        at __randomizedtesting.SeedInfo.seed([CF0D2A58F0EDD5EC:F3B9A537F528CAAC]:0)
        at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:375)
        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:345)
        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:320)
        at app//org.opensearch.sql.datasource.DataSourceAPIsIT.createDataSourceAPITest(DataSourceAPIsIT.java:58)
  2> NOTE: leaving temporary files on disk at: /tmp/tmp7qghdg20/sql/integ-test/build/testrun/integTest/temp/org.opensearch.sql.datasource.DataSourceAPIsIT_CF0D2A58F0EDD5EC-001
  2> org.opensearch.client.ResponseException: method [DELETE], host [http://localhost:9200/], URI [/_plugins/_query/_datasources/create_prometheus], status line [HTTP/1.1 404 Not Found]
    {
      "status": 404,
      "error": {
        "type": "DataSourceNotFoundException",
        "reason": "There was internal problem at backend",
        "details": "Datasource with name: create_prometheus doesn\u0027t exist"
      }
    }
        at __randomizedtesting.SeedInfo.seed([CF0D2A58F0EDD5EC]:0)
        at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:375)
        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:345)
        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:320)
        at app//org.opensearch.sql.datasource.DataSourceAPIsIT.deleteDataSourcesCreated(DataSourceAPIsIT.java:35)
  2> REPRODUCE WITH: ./gradlew ':integ-test:integTest' --tests "org.opensearch.sql.datasource.DataSourceAPIsIT" -Dtests.seed=CF0D2A58F0EDD5EC -Dtests.security.manager=false -Dtests.locale=en-US -Dtests.timezone=UTC -Druntime.java=17

@vamsi-amazon is something missing in Jenkins CI config?

[vamsi-amazon]

This requires plugins.query.datasources.encryption.masterkey to be set in opensearch.yml.
Yes, I set this config in local integTest cluster. we need to set this while running in Jenkins CI config.

[peterzhuamazon]

@vamsi-amazon you can update that config here like this: https://github.com/opensearch-project/opensearch-build/blob/main/manifests/2.9.0/opensearch-2.9.0-test.yml#L129

[vamsi-amazon]

Thanks @peterzhuamazon Raised PR: opensearch-project/opensearch-build#3738

[peterzhuamazon]

Above PR merged.