Update (May 2026): When I moved chat inference onto the same 5090, qwen3-embedding:8b no longer fit alongside it, and I switched to nomic-embed-text with OpenClaw’s native Ollama provider. Details in Running Qwen3.6-35B-A3B Locally on a 5090.

Why I Changed It

I run OpenClaw memory search locally in my homelab. The original setup used a small local embedding model on CPU, which worked but missed semantic matches more often than I wanted.

Goal: keep everything private/local, but improve retrieval quality.

Original Setup

  • OpenClaw memory search provider: local (embeddinggemma-300m)
  • Vector store: sqlite-vec
  • Host: Intel NUC (Arch Linux)

This was cheap and private, but recall quality was limited on nuanced queries.

New Architecture

I moved embeddings to a dedicated GPU server running Ollama:

  • GPU: RTX 5090 (32GB)
  • Embedding model: qwen3-embedding:8b
  • Endpoint: http://<ollama-host>:<port>/v1

So now OpenClaw uses high-quality local embeddings from the dedicated GPU host.

What Broke First (and Why)

I initially tested mxbai-embed-large. Great model, but it failed in this pipeline with:

  • input length exceeds context length

In my case, some indexed chunks exceeded that model’s context limit for embedding requests.

Switching to qwen3-embedding:8b solved it because it has a much larger context window and handles long chunks cleanly.

Config Pattern

The working pattern in openclaw.json (under agents.defaults.memorySearch) is:

1
2
3
4
5
6
7
8
{
  "provider": "openai",
  "model": "qwen3-embedding:8b",
  "remote": {
    "baseUrl": "http://<ollama-host>:<port>/v1",
    "apiKey": "ollama"
  }
}

Then reindex:

1
openclaw memory index --force

Validation

I validated with OpenClaw logs and memory search output:

  • Provider showed as openai (OpenAI-compatible API over Ollama)
  • Model showed as qwen3-embedding:8b

Search quality improved immediately on real memory queries (better first-hit relevance and fewer misses).

Power / Cost Reality

With the embedding model resident in VRAM, nvidia-smi showed ~28W idle draw.

  • 28W x 24h x 30 days ~ 20.16 kWh/month
  • At $0.12/kWh, that’s ~$2.42/month

For this use case, that’s a very reasonable trade for materially better retrieval quality.

Takeaways

  1. Local-first memory search can be both private and high quality.
  2. Embedding model context window matters more than benchmark screenshots.
  3. Homelab GPUs are very practical for always-on retrieval infrastructure.

If you’re already running Ollama on a capable GPU, this upgrade is straightforward and worth doing.