F1 · Edge AI & Agents
DGX Spark RAG Pipeline
Overview
On an NVIDIA DGX Spark, this guide uses e5-base-v2, FAISS, Llama 3.1 8B and a llama.cpp REST server to build a local RAG pipeline from document cleaning, chunking, embedding and indexing to retrieval and generation. The workflow places document processing and vector retrieval on the Grace CPU and runs LLM generation on the Blackwell GPU, producing answers grounded in a sample corpus. This reference implementation chains Hugging Face models, open-source vector search and the Grace–Blackwell unified-memory platform into an observable end-to-end baseline for private-data RAG.
Scope of verification
- It is a local RAG reference workflow on a single DGX Spark, using a specified corpus, embedding model, FAISS index and Llama 3.1 8B to validate the retrieval-to-generation path.
- Dependencies include the Grace Blackwell GB10, a working llama.cpp REST server, e5-base-v2, FAISS, model and document assets, and consistent chunking, indexing and retrieval parameters.
- Scope centers on end-to-end functional integration; it does not cover retrieval-quality evaluation, data permissions, prompt-injection defense, incremental updates, observability, HA, long-run reliability or production RAG operations.
Canonical source: https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/
This page is a guided summary with boundary notes, not an official reproduction.
This page is a guided summary with boundary notes, not an official reproduction.