Skip to content

Commit

Permalink
always set n_ctx to n_ctx_train
Browse files Browse the repository at this point in the history
  • Loading branch information
AsbjornOlling committed Nov 26, 2024
1 parent 17c31d3 commit 0638430
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions nobodywho/src/llm.rs
Original file line number Diff line number Diff line change
@@ -73,6 +73,7 @@ pub fn run_worker(
let n_threads = std::thread::available_parallelism().unwrap().get() as i32;
let ctx_params = LlamaContextParams::default()
.with_seed(seed)
.with_n_ctx(std::num::NonZero::new(model.n_ctx_train()))
.with_n_threads(n_threads)
.with_n_threads_batch(n_threads);

0 comments on commit 0638430

Please sign in to comment.