Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CDK: make default_paginator.page_token_option optional #19368

Merged
merged 4 commits into from
Nov 14, 2022
Merged

CDK: make default_paginator.page_token_option optional #19368

merged 4 commits into from
Nov 14, 2022

Conversation

grubberr
Copy link
Contributor

@grubberr grubberr commented Nov 13, 2022 •
edited
Loading

Signed-off-by: Sergey Chvalyuk grubberr@gmail.com

What

There are cases where we want calculate pagination token without injecting them to requests params.
Make page_token_option and page_size_option optional.

paginator:
      type: DefaultPaginator
      url_base: "*ref(definitions.requester.url_base)"
      pagination_strategy:
        type: "OffsetIncrement"
        page_size: 50
      # page_token_option:
      #  field_name: page_token
      #  inject_into: request_parameter
      # page_size_option:
      #  inject_into: request_parameter
      #  field_name: page_size

We can use next_page_token later in requester.request_options_provider.request_parameters:

requester:
    request_options_provider:
      request_parameters:
        limit: "'{{ next_page_token['next_page_token'] or '0' }},50'"

@grubberr
make page_size_option, page_token_option - optional
54d0965 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr grubberr requested a review from a team as a code owner November 13, 2022 09:36
@grubberr grubberr self-assigned this Nov 13, 2022
@octavia-squidington-iv octavia-squidington-iv added the CDK Connector Development Kit label Nov 13, 2022
@grubberr grubberr requested a review from girarda November 13, 2022 09:53
@grubberr grubberr mentioned this pull request Nov 13, 2022
Merged
14 tasks
girarda
girarda approved these changes Nov 14, 2022
View reviewed changes
Copy link
Contributor

@girarda girarda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @grubberr !

@grubberr
Merge branch 'master' into grubberr/cdk-declarative-default_paginator…
fee0cdb 
…-fix
@grubberr
bump 0.9.2
4c29c2d 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr
bump 0.9.2
3ea3073 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr
Copy link
Contributor Author

grubberr commented Nov 14, 2022 •
edited by github-actions bot
Loading

/publish-cdk dry-run=true

🕑 https://github.com/airbytehq/airbyte/actions/runs/3463950299
✅ https://github.com/airbytehq/airbyte/actions/runs/3463950299

@grubberr
Copy link
Contributor Author

grubberr commented Nov 14, 2022 •
edited by github-actions bot
Loading

/publish-cdk dry-run=false

🕑 https://github.com/airbytehq/airbyte/actions/runs/3464017858
✅ https://github.com/airbytehq/airbyte/actions/runs/3464017858

@grubberr grubberr merged commit d5b66e9 into master Nov 14, 2022
@grubberr grubberr deleted the grubberr/cdk-declarative-default_paginator-fix branch November 14, 2022 18:16
@github-actions github-actions bot mentioned this pull request Nov 15, 2022
Closed
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Nov 18, 2022
Merged
akashkulk pushed a commit that referenced this pull request Dec 2, 2022
@grubberr @akashkulk
CDK: make default_paginator.page_token_option optional (#19368)
dfa4948 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@lazebnyi lazebnyi linked an issue Dec 9, 2022 that may be closed by this pull request
CDK: make default_paginator.page_token_option optional #20294
Closed
@lazebnyi lazebnyi mentioned this pull request Dec 9, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@girarda girarda girarda approved these changes

Assignees

@grubberr grubberr

Labels
CDK Connector Development Kit
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CDK: make default_paginator.page_token_option optional
3 participants
@grubberr @girarda @octavia-squidington-iv