Skip to content

v0.6.4 - HF more caching for no-batch module

Compare
Choose a tag to compare
@kddubey kddubey released this 27 Oct 18:06
· 161 commits to main since this release

Breaking changes

  • The default batch_size in cappr.huggingface is now 2, not 32
  • The implementation for cappr.huggingface.classify_no_batch is now in cappr.huggingface.classify_no_batch_no_cache

New features

  • cappr.huggingface.classify_no_batch now caches the prompt, which makes it much faster. It can also cache shared instructions or exemplars for prompts using the new context manager. See this functionality in action in the Banking 77 demo

Bug fixes

None