How to use Ollama: run AI models on your own machine

Ollama is the shortest path from “I want to run an AI model on my own machine” to a model actually running. It is one install command and one run command, it is free and open source under the MIT license, and once it is up it quietly serves an API on your laptop that your editor and your agents can talk to. For a vibe coder watching the meter, that is the appeal in one line: local inference costs nothing per token and has no rate limit except your hardware.

Here is the whole answer up front. Install Ollama, run ollama run qwen2.5-coder, and you have a coding model answering on localhost:11434. Point an OpenAI-compatible editor or agent at that address and you are coding against a model that never leaves your machine. The two catches, both covered below: your RAM sets the ceiling on how good a model you can run, and Claude Code will not connect to it without a translation layer.

Install it: one command

Verified from Ollama’s own README on 12 August 2026. Pick your OS:

• macOS and Linux: curl -fsSL https://ollama.com/install.sh | sh
• Windows: irm https://ollama.com/install.ps1 | iex, or download the installer from ollama.com/download

That installs the ollama command and a background service. There is nothing else to configure to get started; the service is what serves the local API you will use later.

The commands you actually need

Ollama’s CLI is small enough to learn in a minute. These are the ones you will use daily:

• ollama run <model> downloads the model if needed and starts a chat with it. ollama run qwen2.5-coder is a fine first command.
• ollama pull <model> downloads a model without running it, for later.
• ollama list shows what you have downloaded, and their sizes on disk.
• ollama ps shows what is currently loaded in memory.
• ollama rm <model> deletes a model you are done with, because they are large.

Everything else is a variation on those. The model names come from ollama.com/library, and a name can carry a size tag, so qwen2.5-coder:14b pulls the 14-billion-parameter version specifically.

Which coding model, by your RAM

This is the question the search results are full of, and the answer is that your memory decides it. A model has to fit in RAM (or GPU VRAM) to run at a usable speed, and a rough rule is that a quantized model needs a bit more than its parameter count in gigabytes. So:

• 8 GB of RAM or VRAM: run a 7B-class model, such as qwen2.5-coder:7b. Capable for autocomplete and small edits.
• 16 GB: a 14B model like qwen2.5-coder:14b fits and is a real step up for reasoning about code.
• 32 GB or more: you can run a 32B-class model, though it will be slower; this is where local starts to feel genuinely useful.

The code-tuned models worth knowing, all in the library, are Qwen2.5-Coder and Qwen3-Coder, DeepSeek-Coder-V2, CodeLlama, CodeGemma, and StarCoder2. The one thing no local model will do is match a frontier hosted model on the hardest tasks, which is the trade you are making: you swap some capability for zero cost and full privacy. Whether that trade is worth it for your work is exactly the calculation our cost study frames.

Wire it into your editor and agents

A running model is only useful once your tools can reach it, and this is where Ollama earns its place. The background service exposes two APIs on localhost:11434: its native one at /api/chat and /api/generate, and an OpenAI-compatible one at /v1 (for example /v1/chat/completions), where the API key can be any placeholder like ollama.

That OpenAI-compatible endpoint is the important part, because most agents already speak the OpenAI API. Point one at http://localhost:11434/v1 and it just works: OpenCode, Aider, Cline, and Zed all connect to a local Ollama model directly this way, which the agent field guide covers as the model-agnostic camp. That is the whole setup: pick the agent, set its base URL to your local address, name your model, and you are running an agent against a model on your own laptop.

The exception is the one people search for most: Claude Code will not point at Ollama by itself. Claude Code speaks Anthropic’s API, not the OpenAI one, so connecting it to a local model needs a translation proxy in between (a router or a tool like LiteLLM that presents an Anthropic-shaped endpoint and forwards to Ollama). It is possible, it is not native, and if your goal is simply “an agent using a local model,” reaching for OpenCode or Aider is the shorter road than bending Claude Code to it.

When local is not enough: Ollama Cloud

Some models are too large for any laptop, and Ollama’s answer is Ollama Cloud, which runs big models on their hardware while you keep the same commands. You ollama signin, then pull a cloud model by its -cloud suffix, such as ollama pull gpt-oss:120b-cloud, and run it through the exact same CLI, Python, JavaScript, or API you already use. There is a free tier with usage limits, and the appeal is that a 120-billion-parameter model runs from the same ollama run muscle memory as a 7B one on your machine. It is the escape hatch for the days your hardware is the bottleneck, without leaving the tool.

Ollama versus the alternatives

Two comparisons come up constantly, and both have short answers. LM Studio does much of what Ollama does with a graphical interface instead of a command line, so it suits people who would rather click than type; Ollama suits people who want a command and an API to script against. llama.cpp is the lower-level engine that does the actual model running, and Ollama is a friendlier layer on top of that lineage, trading some fine control for a much simpler setup. None of these is wrong; they sit at different points on the same convenience-versus-control line, and the vibe coding stack places where each fits.

Is it free, and the real limit

Yes. Ollama is MIT-licensed and running models locally is free, unlimited, and fine for commercial use; there is no per-token bill because there is no vendor counting tokens. The cost you actually pay is threefold and worth naming: the hardware to run a decent model, the electricity while it runs, and the capability gap against a frontier hosted model on hard problems. For the large, cost-conscious slice of coding that is autocomplete, refactors, and boilerplate, a local model on Ollama is genuinely enough and genuinely free, which is why it belongs in the toolkit even if a hosted model still does your hardest thinking. Start with ollama run qwen2.5-coder, wire it into the agent you already use, and you will know within an afternoon whether your machine is fast enough to make it a habit.