Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simple request URI modification logic #311

Merged
merged 2 commits into from
Apr 22, 2020

Conversation

awiejacha
Copy link
Contributor

This small code gives opportunity to 'intercept' and modify request URI on-the-fly. When application is used to crawl/visit over resources, making request in the automated way, in some cases it is needed to modify uri/querystring, basing on some rules - e.g. we know that for specific location additional querystring or chunk of path needs to be appended/removed.

Additionally, fixed a bug with resp.cookies being undefined if follow_set_cookies is enabled and previous response cookies not present and set-cookie headers is not present.

@tomas
Copy link
Owner

tomas commented Apr 20, 2020

Looks good but I'd try to use a more descriptive method for the option. Something like uri_modifier perhaps?

@awiejacha
Copy link
Contributor Author

awiejacha commented Apr 21, 2020

Sure - just 5 chars more and much more description.

@tomas
Copy link
Owner

tomas commented Apr 21, 2020

OK would you update your PR?

@awiejacha
Copy link
Contributor Author

Updated. Thanks for keeping eye on this!

@tomas tomas merged commit 0ef49f6 into tomas:master Apr 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants