Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add env param KV_CACHE_LOCATION to control kv cache memory numanode location #462

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Add env param KV_CACHE_LOCATION to control kv cache memory numanode location #462

wants to merge 4 commits into from

Commits on Jun 28, 2024

  1. Add env param KV_CACHE_LOCATION to control kv cache memory numanode l… 

    …ocation

Usage:
before you run instance
export KV_CACHE_LOCATION=#memory_numa_node_id_you_want_to_use_for_kv_cache

by defaults, kv_cache location is the same as other parts of instance.
    @a3213105
    a3213105 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e80f266 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. move env KV_CACHE_LOCATION init & get into environment.h

    @a3213105
    a3213105 committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    c720606 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. Fix typo

    @a3213105
    a3213105 committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    2b7d50f View commit details
    Browse the repository at this point in the history

  2. Recheck the code

    @a3213105
    a3213105 committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    f0236e6 View commit details
    Browse the repository at this point in the history