Building Local RAG with DeepSeek-R1, Ollama & pgvector: Zero API Cost Intelligence

Building Retrieval-Augmented Generation (RAG) no longer requires sending proprietary business data to external API providers. By combining DeepSeek-R1 reasoning models via Ollama with PostgreSQL pgvector extensions, developers can deploy private, zero-token-cost AI search engines locally.

1. The Stack: Why PostgreSQL + pgvector Wins

Dedicated vector databases add extra operational overhead. Using PostgreSQL with pgvector allows you to keep relational data, full-text keyword search, and 1536-dimensional vector similarity indexes within a single reliable database.

-- Enabling pgvector extension & creating HNSW cosine index
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE document_embeddings (
  id BIGSERIAL PRIMARY KEY,
  content TEXT NOT NULL,
  metadata JSONB,
  embedding vector(1536)
);

CREATE INDEX ON document_embeddings 
USING hnsw (embedding vector_cosine_ops) 
WITH (m = 16, ef_construction = 64);

2. Ingestion & Embedding Pipeline

Generate embeddings locally using Ollama's nomic-embed-text model. This eliminates network latency and avoids API rate limits during mass PDF/Markdown indexing.

3. Hybrid Search: Combining BM25 with Vector Cosine Similarity

Vector similarity excels at semantic concept matching but can miss exact part numbers or function names. Performing Reciprocal Rank Fusion (RRF) between SQL full-text search (to_tsquery) and vector distance (<=>) yields 98%+ retrieval accuracy in production tests.