Amazon Seller Partner Fix: 0 records read in reports

[prudhvi85]

What

All the report streams are returning 0 records

How

source API changed created_since_time to data_start_time

🚨 User Impact 🚨

No breaking changes

[prudhvi85]

@harshithmullapudi @alafanechere. Can you merge this PR? it is kind of urgent for us

[alafanechere]

Thanks for the fix. Please provide a clean changelog. I'm running acceptance tests on a side branch, if they're passing I'll publish the new connector version.

[alafanechere]

@prudhvi85 could you explain why this change fixes a problem for you?
According to the API documentation of the currently used version (2020-09-04) the reports endpoint does not take the dataStartTime parameter and createdSince remains valid.
@lizdeika do you encounter the same problem as @prudhvi85?

[prudhvi85]

@alafanechere

Here is the log when we use createdSince as you can see the datastarttime is same as dataendtime. So it is returning 0 records. There is no createdSince In the response from amazon :

{"type": "LOG", "log": {"level": "INFO", "message": "Starting syncing SourceAmazonSellerPartner"}}
{"type": "LOG", "log": {"level": "INFO", "message": "Syncing stream: GET_FLAT_FILE_ALL_ORDERS_DATA_BY_ORDER_DATE_GENERAL "}}
{"type": "LOG", "log": {"level": "WARN", "message": "There are no report document related in stream `GET_FLAT_FILE_ALL_ORDERS_DATA_BY_ORDER_DATE_GENERAL`. Report body {'reportType': 'GET_FLAT_FILE_ALL_ORDERS_DATA_BY_ORDER_DATE_GENERAL', 'processingEndTime': '2022-01-19T09:44:34+00:00', 'processingStatus': 'CANCELLED', 'marketplaceIds': ['ATVPDKIKX0DER'], 'reportId': '710421019011', 'dataEndTime': '2022-01-19T09:44:16+00:00', 'createdTime': '2022-01-19T09:44:16+00:00', 'processingStartTime': '2022-01-19T09:44:22+00:00', 'dataStartTime': '2022-01-19T09:44:16+00:00'}"}}
{"type": "LOG", "log": {"level": "INFO", "message": "Read 0 records from GET_FLAT_FILE_ALL_ORDERS_DATA_BY_ORDER_DATE_GENERAL stream"}}
{"type": "LOG", "log": {"level": "INFO", "message": "Finished syncing SourceAmazonSellerPartner"}}
{"type": "LOG", "log": {"level": "INFO", "message": "SourceAmazonSellerPartner runtimes:\n"}}
{"type": "LOG", "log": {"level": "INFO", "message": "Finished syncing SourceAmazonSellerPartner"}}

Log when I change createdSince to datastarttime :

{"type": "LOG", "log": {"level": "INFO", "message": "Read 2830 records from GET_FLAT_FILE_ALL_ORDERS_DATA_BY_ORDER_DATE_GENERAL stream"}}
{"type": "LOG", "log": {"level": "INFO", "message": "Finished syncing SourceAmazonSellerPartner"}}
{"type": "LOG", "log": {"level": "INFO", "message": "SourceAmazonSellerPartner runtimes:\n"}}
{"type": "LOG", "log": {"level": "INFO", "message": "Finished syncing SourceAmazonSellerPartner"}}

[prudhvi85]

@alafanechere

request body expected by API :
link : https://github.com/amzn/selling-partner-api-docs/blob/main/guides/en-US/use-case-guides/reports-api-use-case-guide/reports-api-use-case-guide-2020-09-04.md

[alafanechere]

This doc is not consistent with this one. This is why I'm really confused. I hope this is not a special parameter only valid for en-US...

[prudhvi85]

@alafanechere I tried with Amazon India as well. dataStartTime works fine.

[prudhvi85]

@alafanechere I am currently using the local docker image with this fix. Logs from Amazon India. Before the fix, it was returning 0 records.

[prudhvi85]

Marketplace Amazon India. Connector version - 2.12

[alafanechere]

Right @prudhvi85, thank you for testing on the Indian marketplace. I'm publishing the connector now, will merge once it's finish.

[alafanechere]

Thanks for this fix @prudhvi85!

[lizdeika]

Hi @prudhvi85 and @alafanechere,

createdSince and dataStartTime are meant for different purposes.

dataStartTime is for data itselft in the report, used when generating report.
createdSince is for report that was created not earlier than YYYY-MM-DD, used when fetching report.

createdSince parameter is meant for getting generated report and not for generating report(like it was implemented until now).
So the implementation was wrong and those reports were not working from the start when someone implemented them.

We do not use those reports so we never noticed.
In my PR I just extracted report data to a method for child classes to be able to extend and/or overwrite it: d7ef91f#diff-7e8be1cd3e8f6950196d15a45ec1df7af0f761f67aa662aef080db5b33b8c4f6R216

[prudhvi85]

@lizdeika Now it makes sense. Thanks for the clarification.