# What is a local LLM?

A local LLM is a large language model you download and run on your own hardware instead of using ChatGPT or Claude in the browser.

A local LLM is a large language model that runs on your computer or server. You download model weights (often open-source from Hugging Face) and run inference with tools like Ollama, LM Studio, or llama.cpp.

Why bother? Your prompts and documents never leave your machine. You can work offline. You avoid per-token API bills — though you pay in RAM, GPU, and electricity instead.

Tradeoffs: smaller local models may be weaker than frontier cloud models. You need enough VRAM or unified memory — llama.cpp's guidance and Ollama's model library list sizes per quantization. Rough rule: 8 GB+ for usable 7B–8B models, more for larger ones. Setup takes a few minutes but is much easier in 2026 than it was in 2023.



## Sources
1. [Hugging Face — Model Hub](https://huggingface.co/models)
2. [Ollama](https://ollama.com/)
3. [LM Studio](https://lmstudio.ai/)
4. [llama.cpp (GitHub)](https://github.com/ggerganov/llama.cpp)
5. [Ollama model library](https://ollama.com/library)

---
Page: https://localizedaiguide.com/q/what-is-a-local-llm