Unable to query Data Stream[BUG]

[akuzin1]

What is the bug?
Querying a Data stream is successful when query is being executed directly using SQL plugin in OpenSearch Dashboard, however, causes an error when using the JDBC driver.

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Create a Data Stream in Opensearch Domain, following OpenSearch Documentation on Data Streams
   or AWS documentation on managing time-series data in Amazon Opensearch Service
2. Insert data, using the above documentation as example.
3. Execute a query against Data Stream using the JDBC driver.

What is the expected behavior?
Querying Data stream using JDBC driver should result in same behavior as SQL plugin.

What is your host/environment?

• Driver Version 1.3.0.0

Do you have any screenshots?

SELECT "message", "@timestamp" FROM "datalogs-test"
Error Message when using JDBC Driver:

java.lang.RuntimeException: Internal Server Error. Reason: There was internal problem at backend. Type: IllegalArgumentException. Details: Index type [datalogs-test] does not exist
	at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:117)
Caused by: Internal Server Error. Reason: There was internal problem at backend. Type: IllegalArgumentException. Details: Index type [datalogs-test] does not exist
	at org.opensearch.jdbc.StatementImpl.executeQueryRequest(StatementImpl.java:72)
	at org.opensearch.jdbc.PreparedStatementImpl.executeQueryX(PreparedStatementImpl.java:77)
	at org.opensearch.jdbc.PreparedStatementImpl.executeQuery(PreparedStatementImpl.java:68)
	at com.amazonaws.athena.connectors.jdbc.manager.JdbcRecordHandler.readWithConstraint(JdbcRecordHandler.java:160)
	at com.amazonaws.athena.connector.lambda.handlers.RecordHandler.doReadRecords(RecordHandler.java:197)
	at com.amazonaws.athena.connector.lambda.handlers.RecordHandler.doHandleRequest(RecordHandler.java:163)
	at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:147)
	at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:112)\```

[akuzin1]

Hi, just wanted to check in and ask about the status of the following issue? Thanks

[forestmvey]

Does it work if you remove the quotes in the from clause?
SELECT "message", "@timestamp" FROM datalogs-test

[akuzin1]

@forestmvey Removing the quotes results in the same error.

[Yury-Fridlyand]

I reproduced and found the root case. As we discussed in opensearch-project/sql-jdbc#90, a java application which uses JDBC driver sets fetch_size parameter, which activates pagination.

Now, a bit of technical details: SQL plugin has 2 engines, V2 supports paging only select * from <table> queries (released in 2.8) and V1 supports a bit more.
Refs: one, two.

So having a paged query which is more complex than select-star-from-table causes fall back to the legacy engine, which does not support data streams. And it wouldn't.

There are two ways of solving this problem:

1. Don't use pagination (don't set fetchSize or set to 0).
2. Wait for pagination to support more features in V2, it is tracked in ... (I'll update this comment and post GH ticket link here). Note: pagination can't support all SQL plugin features, so it is better to go with 1 if possible.