ArxivAPIWrapper - doc_content_chars_max

[hp0404]

This PR refactors the ArxivAPIWrapper class making doc_content_chars_max parameter optional. Additionally, tests have been added to ensure the functionality of the doc_content_chars_max parameter.

Fixes #6027 (issue)

[dean-boi]

Noobie question perhaps: why has this got a limit at all? Should it not default to type None?

[hp0404]

Noobie question perhaps: why has this got a limit at all? Should it not default to type None?

It seems that @dev2049 introduced the doc_content_chars_max parameter with the default 4000 chars value in this PR - I'm not sure why it's 4000 and not some other value (or None), but we can change it too in this PR if needed.

[dean-boi]

Hi again, I made some further changes to the arxiv document loader. I'm not sure what the proceedure is though; should I submit a pull request to your branch [hp0404:arxiv-doc-content-chars] or to [hwchase17:master]?

[hp0404]

Hi again, I made some further changes to the arxiv document loader. I'm not sure what the proceedure is though; should I submit a pull request to your branch [hp0404:arxiv-doc-content-chars] or to [hwchase17:master]?

The best practice in this situation would depend on the development workflow and the specific guidelines the project maintainers set- let's wait for them to respond because I'm not sure as well... e.g.,

• you can submit the pull request to the branch "hp0404:arxiv-doc-content-chars"; this PR gets updated with your commits and possibly merged into the master as a single PR.
• or you could create another/alternative PR directly to the hwchase17:master branch, ignoring this one altogether (e.g. I only fixed the issue related to ArxivAPIWrapper as noted in the original issue while it seems you also updated the Arxiv document loader - should it be a separate PR or should this PR be broader and include both ArxivAPIWrapper and Arxiv document loader updates?)
• or wait for this PR to be merged and propose additional changes in a separate PR to the updated hwchase17:master branch (that will include this PR)
• etc.