[Backport 2.x] Fixed bulk index requests in BWC tests and hardened assertions

[nibix]

Description

Backported from #4817

So far, the BWC test SecurityBackwardsCompatibilityIT.testDataIngestionAndSearchBackwardsCompatibility() did not ingest any documents, as the submitted bulk requests were invalid.

The test already asserted that _bulk API calls returned a 200 OK HTTP status. However, this is not sufficient, as the _bulk API also returns a 200 status if processing of the bulk items fails. Such failures are only reflected item-wise in the response body. This indeed happened, as the test erroneously serialized a string containing the JSON document again as JSON, creating a JSON string containing JSON. This was rejected by OpenSearch.

This is now fixed - additional assertions have been created to verify that the items of the bulk request are actually indexed.

Additionally, the DLS rules of the user would not match any of the ingested documents. This would have caused the _search API always to return empty result sets - even if any documents were ingested.

The indexed test documents were adjusted to have proper genre attributes which can match the configured DLS rule. Additional assertions were added that verify that the DLS and FLS rules are correctly applied to the result set.

• Category: Test fix
• Why these changes are required? - the test did not cover as much functionality as it should
• What is the old behavior before changes and new behavior after changes? - no changes

Testing

• Verified behavior in CI

Check List

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 64.58%. Comparing base (5902191) to head (332828b).
Report is 6 commits behind head on 2.x.

Additional details and impacted files

@@            Coverage Diff             @@
##              2.x    #4831      +/-   ##
==========================================
- Coverage   64.66%   64.58%   -0.08%     
==========================================
  Files         320      320              
  Lines       22372    22380       +8     
  Branches     3611     3616       +5     
==========================================
- Hits        14467    14455      -12     
- Misses       6127     6141      +14     
- Partials     1778     1784       +6     

see 5 files with indirect coverage changes