# Build a simple RAG pipeline with the xAI Grok API

> Answer questions from your own documents: split them into chunks, retrieve the best matches, and have Grok answer with citations via the xAI API.

Level: Advanced
Tool: Grok
Category: Building
Time: About 40 minutes
Checked: 2026-10-15
Tools: [xAI API docs](https://docs.x.ai/docs/overview), [xAI Console](https://console.x.ai)
HTML page: https://easyaihowtos.com/guides/build-a-simple-rag-pipeline-with-grok

RAG is how AI apps answer questions from private documents. This guide builds a minimal, working version in Python: chunking, retrieval, and a cited answer from Grok via the xAI API. It avoids extra databases so you can see every moving part.

## What you need

- Python 3.9 or newer
- An xAI API key with billing set up (the API is pay-as-you-go)
- A folder of plain-text or Markdown documents to search

## Steps

### 1. Understand RAG in one paragraph

**RAG (retrieval-augmented generation)** means: before asking the model, you **retrieve** the few passages from your documents that best match the question, and put them in the prompt. The model then answers from those passages instead of from memory, which cuts made-up answers and lets you cite sources. This guide uses simple keyword-based retrieval (TF-IDF) so it runs with no extra services.

### 2. Install and set your key

xAI's API is compatible with the OpenAI SDK, so you use that library with xAI's address.
```bash
python3 -m venv .venv && source .venv/bin/activate
pip install openai scikit-learn
export XAI_API_KEY="your-key-here"
```

### 3. Split documents into chunks

Models work best with short, focused passages. A **chunk** is a slice of a document, here about 150 words, overlapping a little so sentences aren't cut off from context.
```python
from pathlib import Path

def load_chunks(folder="docs", size=150, overlap=30):
    chunks = []
    for path in Path(folder).glob("*.md"):
        words = path.read_text().split()
        for start in range(0, len(words), size - overlap):
            text = " ".join(words[start:start + size])
            chunks.append({"source": f"{path.name}#{start}", "text": text})
    return chunks
```

### 4. Retrieve the best chunks

**TF-IDF** scores words that are frequent in a chunk but rare overall, and **cosine similarity** measures how closely the question and each chunk match.
```python
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

chunks = load_chunks()
vectorizer = TfidfVectorizer(stop_words="english")
matrix = vectorizer.fit_transform([c["text"] for c in chunks])

def retrieve(question, k=4):
    scores = cosine_similarity(vectorizer.transform([question]), matrix)[0]
    best = scores.argsort()[::-1][:k]
    return [chunks[i] for i in best if scores[i] > 0]
```

### 5. Ask Grok with the retrieved context

Send the passages and the question to the xAI Responses API. The prompt tells Grok to answer only from the context and cite sources.
```python
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["XAI_API_KEY"], base_url="https://api.x.ai/v1")

def answer(question):
    hits = retrieve(question)
    if not hits:
        return "I couldn't find that in the documents."
    context = "\n\n".join(f"[{h['source']}]\n{h['text']}" for h in hits)
    prompt = (
        "Answer the question using ONLY the context below. "
        "Cite sources in square brackets. If the answer isn't there, say so.\n\n"
        f"Context:\n{context}\n\nQuestion: {question}"
    )
    response = client.responses.create(model="grok-4.7", input=prompt)
    return response.output_text

print(answer("What is our refund window?"))
```

### 6. Test with questions you know the answers to

Write 10 questions with known answers, including 2 that the documents don't cover. Check that answers are right, cite the right file, and say "not in the documents" when they should. If retrieval misses, try smaller chunks or a larger `k`.

## Tips

- When keyword search isn't enough (synonyms, paraphrases), swap TF-IDF for embeddings, which match by meaning. The rest of the pipeline stays the same.
- Keep the model name in one constant. xAI's models page lists current names; `grok-4.7` was the recommended model when this was checked.
- Log the retrieved sources for every answer. Most RAG bugs are retrieval bugs.

## FAQ

### Why not paste all my documents into the prompt?

For a few short files, you can. As documents grow, retrieval keeps prompts smaller, cheaper, and more focused.

### Can I use Claude or ChatGPT instead of Grok?

Yes. Only the final call changes. The chunking and retrieval code works with any model.

### Does Grok search X or the web here?

Not unless you enable xAI's search tools. In this pipeline Grok sees only the context you send.

---

This guide is from Easy AI How-To (https://easyaihowtos.com). It is independent of the tools named above. Screens and free-plan limits change, so check the date before you rely on a button name.