Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for resumption tokens #11

Merged
merged 4 commits into from
Apr 4, 2022
Merged

Conversation

chrispenny
Copy link
Member

@chrispenny chrispenny commented Apr 1, 2022

Resumption Token

Resumption Tokens are a form of pagination, however, they also contain a level of validation.

Each Resumption Token should represent a specific request, including whatever filters might have been applied as part of that request, as well as representing a particular "page" in the Paginated List.

The goal is to increase reliability of pagination by making sure that each requested "page" came from a request containing the expected filters. EG: You can't send an unfiltered request for OAI Records, see that there are 10 pages, and then decide to request page=10 with some filters now applied. The Token itself would be aware that a different filter has been applied, and it would be invalid.

Consistency in config names

I realised that we had some snake case and camel case names. Switched everything to snake case to be consistent with SS framework.

09b0b38

@chrispenny chrispenny force-pushed the feature/resumption-token branch 4 times, most recently from 0e00591 to 4d6af89 Compare April 3, 2022 20:56
@chrispenny chrispenny force-pushed the feature/resumption-token branch from 4d6af89 to da49e46 Compare April 3, 2022 21:08
@chrispenny chrispenny marked this pull request as ready for review April 3, 2022 21:10
Copy link
Contributor

@MelissaWu-SS MelissaWu-SS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! I tested it on my local for the new support for resumptionToken, got 5 results per request, and there is a resumptionToken element in the result if there are more records.

I just noticed that there is a rule for the resumptionToken element when I read the doc.
Screen Shot 2022-04-04 at 11 48 45 AM
I am wondering if we need to implement this? As I found there is no empty resumptionToken element in the last response. 👀

Other than that, all looks great to me! 🌟

@chrispenny
Copy link
Member Author

chrispenny commented Apr 4, 2022

Nice spotting, @MelissaWu-SS! Thanks for picking that up. I'll work to add that now.

@chrispenny chrispenny force-pushed the feature/resumption-token branch from 645e3fd to 039fa8f Compare April 4, 2022 01:28
@chrispenny
Copy link
Member Author

@MelissaWu-SS I've added the empty Resumption Token for the "last page" of a paginated response.

I was also originally not going to bother with expiring Resumption Tokens, however, I've had a change of heart. I've decided that I don't like the idea of Resumption Tokens "living forever".

I've added some new functionality, which is the ability for us to expire our Tokens. The default is 1 hour.
039fa8f

Easiest way to test the token expiry is to get your first response and grab the Resumption Token from it.
Screen Shot 2022-04-04 at 1 31 19 PM

Take it to a Base 64 decoder/encoder (https://www.base64encode.org/), and decode it.
Screen Shot 2022-04-04 at 1 28 04 PM

Change the expiry date to "an hour earlier", re-encode it, and use that as the Resumption Token.
Screen Shot 2022-04-04 at 1 28 30 PM

Copy link
Contributor

@MelissaWu-SS MelissaWu-SS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet! Thanks for the suggestion around manually making the token expired! Otherwise I will probably wait for one hour😀

Just one minor change, other than that, the adding expiry attribute and empty resumption token element are looking great to me!🌟

$resumptionTokenElement = $this->findOrCreateElement('resumptionToken', $listRecordsElement);

$resumptionTokenElement->nodeValue = $resumptionToken;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!$resumptionToken) {
return;
}

Can we please add this for the empty resumption token scenario? 👀
I just realised it would cause an error in the decode process($decode = base64_decode($resumptionToken, true);) if we set an empty resumption token in the last page of the response. Looks base64_decode are not able to decode an empty string.

Screen Shot 2022-04-04 at 2 30 40 PM

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @MelissaWu-SS!

Fixed at the source of the issue:
fad9151

@chrispenny chrispenny merged commit 084c15c into main Apr 4, 2022
@chrispenny chrispenny deleted the feature/resumption-token branch April 4, 2022 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants