Skip to content
This repository has been archived by the owner on May 28, 2023. It is now read-only.

Add support for summarization of longer texts. #17

Open
michalharakal opened this issue Oct 5, 2021 · 0 comments
Open

Add support for summarization of longer texts. #17

michalharakal opened this issue Oct 5, 2021 · 0 comments

Comments

@michalharakal
Copy link

Usually are summarization models trained by text blocks with limited size, e.g. 800 characters for mt5-small-sum-de-en-v1.
To achieve the best prediction performance, also the input text has to be limited to this length.

As one of solution for summarizing of longer texts, following approach could be tested:

  • split input text for prediction into blocks on sentences level, where blocks character count is limited
  • predict a summary for individual block
  • join results of blocks summary into single text block.
michalharakal added a commit to michalharakal/transformer-tools that referenced this issue Oct 5, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant