Skip to content

@Infini-AI-Lab Infini-AI-Lab

Change the repository type filter

All

    Repositories list

    13 repositories

    • MagicPIG

      Public
      MagicPIG: LSH Sampling for Efficient LLM Generation
      decodinglsh-algorithmllm-inferencegpu-cpu
      Python
      •
      Apache License 2.0
      •6•137•2•0•Updated Dec 16, 2024Dec 16, 2024

    • S2FT-Page

      Public
      JavaScript
      •0•0•0•0•Updated Dec 10, 2024Dec 10, 2024

    • S2FT

      Public
      Python
      •2•8•0•0•Updated Dec 9, 2024Dec 9, 2024

    • MagicDec

      Public
      Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
      Python
      •
      Apache License 2.0
      •4•100•6•0•Updated Dec 4, 2024Dec 4, 2024

    • MagicPIG-Page

      Public
      JavaScript
      •1•0•0•0•Updated Dec 2, 2024Dec 2, 2024

    • Factor

      Public
      0•1•0•0•Updated Nov 7, 2024Nov 7, 2024

    • MagicDec-part1

      Public
      Speculative decoding for high-throughput long-context inference
      JavaScript
      •
      Apache License 2.0
      •0•0•0•0•Updated Sep 10, 2024Sep 10, 2024

    • Sirius

      Public
      Sirius, an efficient correction mechanism, which significantly boosts Contextual Sparsity models on reasoning tasks while maintaining its efficiency gain.
      Python
      •4•20•0•0•Updated Sep 10, 2024Sep 10, 2024

    • MagicDec-part2

      Public
      MagicDec: Breaking the Latency-Throughput Tradeoff for Long Contexts with Speculative Decoding
      JavaScript
      •
      Apache License 2.0
      •0•0•0•0•Updated Sep 5, 2024Sep 5, 2024

    • TriForce

      Public
      [COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
      accelerationefficiencyinferencellmlong-contextllm-inferencespeculative-decoding
      Python
      •13•236•7•0•Updated Aug 31, 2024Aug 31, 2024

    • Sequoia

      Public
      scalable and robust tree-based speculative decoding algorithm
      efficiencyinferencellmspeculative-decoding
      Python
      •36•322•7•3•Updated Aug 13, 2024Aug 13, 2024

    • lm-evaluation-harness

      Public
      A framework for few-shot evaluation of language models.
      Python
      •
      MIT License
      •2k•0•0•0•Updated Jun 10, 2024Jun 10, 2024

    • Sequoia-Page

      Public
      JavaScript
      •1•0•0•0•Updated May 21, 2024May 21, 2024