[BUG] OSIS Filter Not Honored

[Conklin-Spencer-bah]

Describe the bug
The AWS Managed Data Prepper (OSIS) Does not seem to consistently honor the filtering that I have configured.

To Reproduce
Steps to reproduce the behavior:

1. Enable Security Lake with cloudtrail and eks audit logs
2. Create Ingestion for Security Lake -> OpenSearch
3. Update pipeline to include routes

  route:
    - eks-logs: '/metadata/product/name == "amazon_eks"'
    - cloudtrail-logs: '/metadata/product/name == "cloudtrail"'

5. Have the following sink configuration

sink:
    - opensearch:
        # Provide an AWS OpenSearch Service domain endpoint
        hosts: [ <redacted> ]
        routes: [eks-logs]
        aws:
          # Provide a Role ARN with access to the domain. This role should have a trust relationship with osis-pipelines.amazonaws.com
          sts_role_arn: "<redacted>"
          # Provide the region of the domain.
          region: "us-east-1"
          # Enable the 'serverless' flag if the sink is an Amazon OpenSearch Serverless collection
          serverless: false
        index: "ocsf-${/metadata/version}-${/class_uid}-${/class_name}-eks-${/accountid}-%{yyyy.MM.dd}"
    - opensearch:
        # Provide an AWS OpenSearch Service domain endpoint
        hosts: [ <redacted> ]
        routes: [cloudtrail-logs]
        aws:
          # Provide a Role ARN with access to the domain. This role should have a trust relationship with osis-pipelines.amazonaws.com
          sts_role_arn: "<redacted>"
          # Provide the region of the domain.
          region: "us-east-1"
          # Enable the 'serverless' flag if the sink is an Amazon OpenSearch Serverless collection
          serverless: false
        index: "ocsf-${/metadata/version}-${/class_uid}-${/class_name}-cloudtrail-${/accountid}-%{yyyy.MM.dd}"

Expected behavior
To create two indexes. One for 'eks' and one for 'cloudtrail'. Instead what happens is that it creates three indexes; one for eks, one for cloudtrail, and then a third that is unlabeled.

The unlabeled index logs contain 'metadata.product.name', and the product name is either 'cloudtrail' or 'amazon_eks'. This should have been routed to the appropriate route but is not.

Screenshots

Environment (please complete the following information):

• AWS Managed OSIS