[feature request] Adding an option to source documents from Arxiv

[dl423]

For a researcher or a masters/PhD student who wants to explore the literature on a technical topic, it might be helpful to add an option to produce the essay based on Arxiv papers only. This way, the resulting essay can contain much more in-depth information compared to sourcing the information from general-purpose Internet search. Providing an option to filter the the source papers by publication date would probably be helpful as well.

For example, if someone wants to get an overview of recent methodologies and architectures proposed for agentic systems, they could choose to set the essay source to be Arxiv papers published in 2024.

If you decide to implement this feature request, the Arxiv API might come in handy : https://info.arxiv.org/help/api/basics.html

Thanks!

[jaigouk]

I think we can inject it like this within frontend https://github.com/jaigouk/storm_wiki/blob/main/util/search.py#L108

[jaigouk]

with open pr, i added it as an option https://github.com/stanford-oval/storm/pull/123/files#diff-37f221c91338a31be4c31f8c6db06b6af74b5544a75e3486b0e891630bea5194R183