Skip to content
This repository has been archived by the owner on Jan 29, 2024. It is now read-only.

Log info on processed batch for topic-extract on arxiv #589

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

FrancescoCasalegno
Copy link
Contributor

Description

Currently, we are logging info while we extract topic info for all sources but arxvi.

The reason why arxiv is special is that we don't send requests to the server using the arxiv ids one by one, but in batches.

So this PR introduces batch-based logging info for arxiv topic extraction.

Checklist

  • This PR refers to an issue from the issue tracker.
    (if it is not the case, please create an issue first).
  • Unit tests added.
    (if needed)
  • Documentation and whatsnew.rst updated.
    (if needed)
  • setup.py and requirements.txt updated with new dependencies.
    (if needed)
  • Type annotations added.
    (if a function is added or modified)
  • All CI tests pass.

@FrancescoCasalegno FrancescoCasalegno requested review from EmilieDel and jankrepl and removed request for EmilieDel February 25, 2022 15:46
Copy link
Contributor

@jankrepl jankrepl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@EmilieDel EmilieDel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants