F1 · Edge AI & Agents
ERNIE MoE on Armv9
Overview
On the Armv9 CPU of the Radxa Orion O6, this guide deploys ERNIE-4.5 MoE with llama.cpp, validating both standard and Thinking model outputs and observing token-level expert routing. The workflow compares a baseline build with one enabling Armv9 instruction features, using pp512, tg128 and combined tests to show execution differences in prompt processing and token generation. This flow chains the open-source MoE model, llama.cpp and Armv9 optimization into an evaluation baseline that examines both model behavior and platform execution characteristics.
Scope of verification
- It is an ERNIE-4.5 MoE deployment-and-evaluation workflow on a single Radxa Orion O6, validating model output, expert routing and execution results under the specified benchmark parameters.
- Dependencies include Armv9 Linux, the specified llama.cpp and compiler configuration, ERNIE-4.5 model assets, sufficient memory and identical build flags, context and benchmark parameters.
- Scope centers on the specified device, model and pp512/tg128 tests; the performance gains on the page do not apply to all Armv9 or MoE workloads, and it does not cover model quality, power, thermal behavior, long-run serving or production deployment.
Canonical source: https://learn.arm.com/learning-paths/cross-platform/ernie_moe_v9/
This page is a guided summary with boundary notes, not an official reproduction.
This page is a guided summary with boundary notes, not an official reproduction.