Performance of Vulkan backend looks amazing #11918
foldl
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I have just updated chatllm.cpp to use
ggml
from the last commit (0f2bbe6). Performance of Vulkan backend looks amazing: it is much faster than the CUDA backend (in this test to be precious).Command line options:
-m qwen2.5-1.5b.bin -ngl all -t 0 -p "write a quick sort function in python" --max_length 200
Hardware: 2080TI with 22GB.
Beta Was this translation helpful? Give feedback.
All reactions