llama.cpp
Local C/C++ inference and an HTTP server across multiple hardware backends.
Where it stands.
A documented path to local inference and serving. Choose a compatible model, format and backend before treating it as an agent's model runtime.
A desk review of published evidence. We have not benchmarked or audited this software.
Behind the verdict.
Adoption signals
Hardware backends and downstream tools are documented ecosystem signals. Deployment scale was not independently measured.
Developer experience
CLI, server, build guides and troubleshooting material. Model conversion and backend selection can add setup effort.
Maintenance
The repository documents a release process and active contribution workflow. Release cadence and support responsiveness were not audited.
Compatibility & access
C/C++, CLI and REST API, GGUF models and varied CPU/GPU backends. MIT runtime; each model has its own license.
What to watch for.
- Model format and backend compatibility must be checked separately.
- Tool calling and structured output depend on the selected model and configuration.
These are documented constraints or editorial implications of the architecture, not an exhaustive defect list.
Follow the evidence.
01Runtime, backends & MIT license
Documents inference, server and CLI tools, model format, backends and release process.
github.comExplore alongside.
Related by catalog category.
Capabilities may differ.