A high-performance local AI inference engine engineered for Windows-ARM64 Snapdragon platforms.

The aura-sdk is a high-performance local AI inference engine engineered specifically for Snapdragon X Elite and Plus platforms. It enables execution of AI models directly on Qualcomm Hexagon NPUs and Oryon CPUs, bypassing virtualization layers to achieve low-latency execution.

Key Features

• Native Genie Engine: Uses the Qualcomm NPU (Hexagon) for best performance, requiring models in Genie binary format.
• Aura Engine (ORT): ONNX Runtime backend with QNN execution provider support for Oryon ARM64 CPU execution.
• Zero-Copy Shared Memory: Direct memory mappings to Hexagon DSP memory spaces to eliminate CPU-to-NPU serialization overhead.

API Integration

To integrate Aura SDK into your Rust application, add it to your Cargo.toml dependencies:

[dependencies]
aura-sdk = { version = "0.2.1" }

To enable ONNX Runtime support (Aura Engine), include the aura-engine feature:

[dependencies]
aura-sdk = { version = "0.2.1", features = ["aura-engine"] }

Running Inference

Genie Engine (Native NPU)

use std::path::Path;
use aura_sdk::engines::GenieEngine;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config_path = Path::new("phi_3_5_mini_instruct-genie-w4a16-qualcomm/genie_config.json");
    let engine = GenieEngine::new(config_path)?;
    
    engine.query_sync("Explain NPUs in one sentence.", 512, |token| {
        print!("{}", token);
    })?;
    
    Ok(())
}

Aura Engine (ONNX Runtime Mode)

use std::path::Path;
use aura_sdk::engines::AuraEngine;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let model_path = Path::new("Llama-1B-ONNX/onnx/model_q4.onnx");
    let mut engine = AuraEngine::new(model_path, "43")?;
    
    engine.query("Explain NPUs in one sentence.", 512, |token| {
        print!("{}", token);
    })?;
    
    Ok(())
}