You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Today the HTTP Source Connector ignores any sort of pagination token and just passes along the received JSON blob.
Describe the solution you’d like
If there is pagination, the HTTP request connector should iterate through all the pages, combine the blobs, and write the final list to the destination.
See scenario #1 of this user doc for real life use case.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered:
There are numerous pagination strategies out there: limits, pages, etc. It'd be ideal if it were possible for the exact keys used to be specified during configuration.
keu
changed the title
HTTP Source Connector should support pagination.
Source HTTP: Add pagination support.
Mar 24, 2021
keu
changed the title
Source HTTP: Add pagination support.
Source HTTP: Add pagination support
Mar 24, 2021
Tell us about the problem you're trying to solve
Today the HTTP Source Connector ignores any sort of pagination token and just passes along the received JSON blob.
Describe the solution you’d like
If there is pagination, the HTTP request connector should iterate through all the pages, combine the blobs, and write the final list to the destination.
See scenario #1 of this user doc for real life use case.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: