Replies: 2 comments
-
If you check your system resources, do you see it loading on the GPU? If not, you may need to recheck/reinstall the library, as per the readme, to ensure it compiled with GPU support. |
Beta Was this translation helpful? Give feedback.
0 replies
-
The output length was limited due to a default parameter |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
my running code is below . but it's not on gpu.?how to run the server on GPU?the output length is short, it seems that n_ctx does't work
Beta Was this translation helpful? Give feedback.
All reactions