DeepSeek R1 & Llama 3.3 Local Setup Guide (Ollama vs vLLM)

:::info[Quick Answer] Run DeepSeek R1 or Llama 3.3 locally for free with Ollama for terminal chat (ollama run deepseek-r1:8b) or vLLM for an OpenAI-compatible API (python3 -m vllm.entrypoints.openai.api_server). Distilled 8B and 14B quantized models run fine on consumer GPUs with 8–16 GB VRAM. :::

Sending proprietary code to ChatGPT or Claude API means your data leaves your machine. Open-weight models like DeepSeek R1 and Meta’s Llama 3.3 let you run reasoning-grade AI locally — no API bill, no data leakage, no rate limits.

The catch: you need a GPU (or patience on CPU). This guide covers both Ollama for quick local chat and vLLM for serving an API your apps can call. Pair it with our Cursor MCP Server setup and Cursor can talk to your local models directly.

Ollama vs vLLM — Pick Your Engine

graph TD
    Client[Local App / Cursor IDE / Terminal] --> Router{Select Local Engine}
    
    Router -->|Developer Terminal Chat| OllamaEngine[Ollama Engine :11434]
    Router -->|High-Throughput API| vLLMEngine[vLLM Server :8000]
    
    subgraph GPU Hardware VRAM
        OllamaEngine -->|GGUF Quantization| LocalGPU[NVIDIA / Apple Silicon GPU]
        vLLMEngine -->|PagedAttention Memory| LocalGPU
    end
    
    LocalGPU --> Output[Local Reasoning Output]

Ollama if you want to chat in a terminal or wire up a single-user workflow. vLLM if you need an OpenAI-compatible endpoint that multiple clients hit at once. Most solo devs start with Ollama and graduate to vLLM when they need an API.

VRAM Requirements — Don’t OOM Your GPU

Pick the wrong model size and you’ll get out-of-memory errors before the first token streams. Match the model to your hardware:

If you’re on an 8 GB card, stick to DeepSeek R1 8B. Don’t try to force 70B — you’ll spend more time troubleshooting OOM errors than getting work done.

Run DeepSeek R1 with Ollama

Ollama is the fastest path to local AI. One install command, one run command, you’re chatting.

# Install Ollama (Linux/macOS)
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run DeepSeek R1 8B
ollama run deepseek-r1:8b

# Or try the 14B variant if you have the VRAM
ollama run deepseek-r1:14b

First run downloads the GGUF weights (a few GB). After that, you get a >>> prompt streaming reasoning tokens locally. DeepSeek R1 shows its chain-of-thought before the final answer — that’s the model working, not a bug.

When I use Ollama: Quick code review on a flight, testing prompts before sending them to a paid API, or running models on a laptop without setting up a full inference server.

Serve Llama 3.3 with vLLM (OpenAI-Compatible API)

vLLM uses PagedAttention to serve multiple concurrent requests efficiently. If you’re building an app that calls /v1/chat/completions, this is the setup.

pip install vllm

vllm serve meta-llama/Llama-3.3-70B-Instruct-AWQ \
  --quantization awq \
  --port 8000 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.92

Test the endpoint:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.3-70B-Instruct-AWQ",
    "messages": [{"role": "user", "content": "Write a Python script for PostgreSQL backups."}]
  }'

You get streaming JSON chunks back — same format as OpenAI’s API. Point Cursor, LangChain, or any OpenAI SDK client at http://localhost:8000/v1 and it just works.

Heads up: vLLM needs Python 3.10+ and CUDA 12. The 70B model needs serious VRAM — if you’re on 16 GB, stick with smaller AWQ-quantized variants or use Ollama instead.

FAQ

How much VRAM does DeepSeek R1 need?

8B (Q4_K_M): 6 GB. 14B: 10 GB. 32B: 20 GB. These are minimums — leave headroom for context length or you’ll OOM mid-conversation.

Ollama or vLLM — which is faster?

Different jobs. Ollama wins on setup speed — you’re running in 60 seconds. vLLM wins on concurrent throughput when multiple clients hit the same endpoint. For solo terminal chat, Ollama. For an API serving a team, vLLM.

Can I run these on CPU only?

Yes, both Ollama and llama.cpp fall back to CPU. Expect 2–5 tokens/sec instead of 30+ on GPU. Usable for testing, painful for daily work. Budget for a GPU if you’re serious about local inference.

What to Read Next

• Cursor MCP Server Setup Guide — connect Cursor to your local Ollama/vLLM endpoint
• Running Small Language Models on Raspberry Pi 5 — edge deployment when a desktop GPU isn’t an option
• Top 10 Self-Hosted DevOps Tools — host your inference stack on your own VPS