Execution latency and memory bandwidth comparison charts for local LLM inference.

Aura SDK is built to achieve low-latency execution. Below are benchmark runs comparing the Snapdragon X Elite NPU executing Aura w4a16 enclaves against conventional CPU execution.

Time to First Token (TTFT)

The time to first token is critical for conversational user interfaces. This benchmark measures prompt processing latency (in milliseconds) for a 512-token input sequence:

Model	Oryon CPU (ONNX FP32)	Hexagon NPU (Aura w4a16)	Speedup Factor
Llama-3-8B	820 ms	64 ms	12.8x
Phi-3-mini	380 ms	28 ms	13.5x
Mistral-7B	710 ms	55 ms	12.9x

Tokens Per Second (TPS)

Generation throughput measurements during autoregressive decoding runs (measured in tokens per second):

Model	Oryon CPU (ONNX FP32)	Hexagon NPU (Aura w4a16)	Throughput Gain
Llama-3-8B	4.8 tps	34.2 tps	7.1x
Phi-3-mini	12.2 tps	68.5 tps	5.6x
Mistral-7B	5.5 tps	38.0 tps	6.9x

Memory Bandwidth & Thermal Efficiency

By utilizing dynamic 4-bit weight decompression inside the Hexagon HMX tensor units, Aura SDK reduces memory bus traffic. This decreases system power draw from a typical 18W on CPU cores down to 4.2W on NPU channels, preserving battery life and maintaining low thermal load during long inference enclaves.