Report your speed
The fastest way to get comparable numbers is llama.cpp's benchmark: llama-bench -m model.gguf prints prompt processing (pp512) and generation (tg128). With Ollama, run ollama run model --verbose and copy the eval rates.
The fastest way to get comparable numbers is llama.cpp's benchmark: llama-bench -m model.gguf prints prompt processing (pp512) and generation (tg128). With Ollama, run ollama run model --verbose and copy the eval rates.