F2 · Edge AI & Agents
llama.cpp on DGX Spark
Overview
On the Grace–Blackwell platform of an NVIDIA DGX Spark, this guide builds both CUDA and CPU-only llama.cpp and uses a quantized GGUF model to validate GPU inference and Arm CPU execution paths. The workflow observes the Grace CPU's Armv9 SIMD usage through runtime output and instruction analysis, confirming the specified workload mainly uses Neon with no SVE/SVE2 activity. This dual-backend flow chains llama.cpp, the open-source GGUF ecosystem and the DGX Spark heterogeneous platform into a baseline for checking backend portability and CPU execution behavior.
Scope of verification
- It is an llama.cpp backend reference workflow on a single DGX Spark, validating that CUDA and CPU-only builds run the specified quantized model and observing the CPU workload's SIMD instruction activity.
- Dependencies include the Grace Blackwell GB10, a matching NVIDIA software stack, the llama.cpp version, CUDA/CPU build configuration, a quantized GGUF model and consistent runtime parameters.
- Scope centers on backend execution and instruction-activity observation; the Neon/SVE results apply only to the specified workload, do not constitute a CPU–GPU hybrid benchmark, and do not cover other kernels, models, power, thermal behavior or production serving.
Canonical source: https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/
This page is a guided summary with boundary notes, not an official reproduction.
This page is a guided summary with boundary notes, not an official reproduction.