kitio.aiAdded Oct 28, 2026
llama.cpp
FreeC/C++ engine for running LLMs efficiently on CPUs and GPUs
What is llama.cpp?
llama.cpp is an open-source C/C++ inference engine that runs large language models efficiently on laptops, desktops and servers, with or without a GPU. It created the GGUF model format and powers many local AI apps, including LM Studio and Jan.
It supports quantization from 2 to 8 bits, Apple Metal, CUDA, Vulkan and ROCm backends, and a built-in server with an OpenAI-compatible API and web UI. New open models usually gain support within days of release.
The project is free under the MIT license and maintained by Georgi Gerganov and a large community under the ggml-org organization, with prebuilt binaries for major platforms.
Features
- GGUF format
- Low-bit quantization
- Many GPU backends
- OpenAI-compatible server
- Built-in web UI
Tags
Releases
+ Submit new release- Listed
llama.cpp was added to the directory.
Organization
ggml-org
Listed by kitio.ai · unverified owner
Pricing
- Pricing model
- Free
- Paid options from
- 100% free
Sponsored
Your AI tool here
A Featured launch puts your tool in this spot and on the home page for a week.
Top llama.cpp alternatives
View allModels & APIs
Run open models locally with one command, plus optional cloud models
Models & APIs
High-throughput open-source engine for serving LLMs in production
Models & APIs
Apple's array framework for running and training models on Apple Silicon
If you liked llama.cpp
Gradio web interface for running local LLMs with many backends
FreeModels & APIsFast open-source serving framework for LLMs and vision models
FreeModels & APIsMade in United StatesPackage an LLM and its runtime into one portable executable file
FreeModels & APIsMade in United States
Pros and cons
Pros
- Runs on modest hardware
- Very fast new-model support
- Foundation of many apps
Cons
- Command-line heavy
- Build options can confuse
- Less suited to high-concurrency serving
Prompts & results
No prompts yet. Add your own prompt and output to help others use this tool.
Share a prompt
Reviews
Q&A
What is llama.cpp?
llama.cpp is an open-source C/C++ inference engine that runs large language models efficiently on laptops, desktops and servers, with or without a GPU. It created the GGUF model format and powers many local AI apps, including LM Studio and Jan.
How much does llama.cpp cost?
llama.cpp is 100% free.
Is there a free version of llama.cpp?
Yes. llama.cpp is completely free to use.
What are the main features of llama.cpp?
GGUF format, Low-bit quantization, Many GPU backends, OpenAI-compatible server, Built-in web UI.
Who makes llama.cpp?
llama.cpp is made by ggml-org.
What are the best alternatives to llama.cpp?
Popular alternatives include KoboldCpp, Ollama, vLLM.