NexaAI /nexa-sdk

Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

★ 8,489⑂ 1,075👁 531⑃ 773● 288▤ 78
+0
Daily
+0
Weekly
+49
Monthly
8,489
stars
1,075
forks
531
watchers
773
prs
288
issues
78
releases

README

The easiest way to run frontier LLMs & VLMs locally on Qualcomm devices







Documentation · Quickstart · Models · Community



---

GenieX is an on-device Gen AI inference runtime for Qualcomm devices. Bring almost any GGUF model from Hugging Face, ModelScope, or Docker Hub, or use a pre-compiled bundle from Qualcomm AI Hub. Run it locally on the Hexagon NPU, Adreno GPU, or CPU in a few lines of code. One C SDK underneath is exposed through a CLI, Python, Kotlin/Java, Docker, and an OpenAI-compatible server. It is the community version of Qualcomm GENIE.


  


 Supported platforms

GenieX runs only on Qualcomm Snapdragon. Find your platform, then jump straight to the interface you want to use.

| Platform | Example devices | Jump to a quickstart |
| --- | --- | --- |
| 🪟 Windows ARM64 (Compute) | Snapdragon X · X Elite | CLI · Python · Local server |
| 🤖 Android (Mobile) | Snapdragon 8 Elite · 8 Elite Gen 5 | Android SDK |
| 🐧 Linux ARM64 (IoT) | Dragonwing QCS9075 | CLI · Docker · Python |


 No device on hand? Spin up a remote session on Qualcomm Device Cloud.

---

 Quickstart

Pick your interface below. Each one follows the same three steps: Install, Run, and Docs. The examples show both runtimes: a GGUF model from Hugging Face, ModelScope, or Docker Hub (llama_cpp) and a pre-compiled bundle from Qualcomm AI Hub (qairt, NPU).

 CLI

!Windows ARM64 !Linux ARM64

Install

- Windows ARM64 — download the installer, run it, then open a new terminal.
- Linux ARM64 — one line, no sudo:
  bash
  curl -fsSL https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-geniex/install.sh | sh
  

Run — chat with any model in one line (drag in an image for VLMs):

bash
 GGUF from Hugging Face → llama.cpp (NPU / GPU / CPU)
geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf

 GGUF from ModelScope → llama.cpp (NPU / GPU / CPU)
geniex infer https://modelscope.cn/models/Qwen/Qwen3-0.6B-GGUF

 Pre-compiled bundle from Qualcomm AI Hub → Qualcomm AI Engine Direct (NPU)
geniex infer ai-hub-models/Qwen2.5-VL-7B-Instruct

 GGUF from Docker Hub (https://hub.docker.com/u/ai) → llama.cpp (NPU / GPU / CPU)
geniex infer docker.io/ai/gemma3


📖 Docs — Install · Quickstart · Command reference

 Python

!Windows ARM64 !Linux ARM64

Install

bash
pip install geniex


Run — mirrors Hugging Face transformers (from_pretrained() → .generate()):

python
 GGUF from Hugging Face → llama.cpp
from geniex import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3.5-2B-GGUF", precision="Q4_0")

messages = [{"role": "user", "content": "What is 2+2?"}]
prompt = model.tokenizer.apply_chat_template(messages, add_generation_prompt=True)

for chunk in model.generate(prompt, max_new_tokens=256, stream=True):
    print(chunk, end="", flush=True)

model.close()


python
 Pre-compiled bundle from Qualcomm AI Hub → Qualcomm AI Engine Direct (NPU)
from geniex import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("ai-hub-models/Qwen3-4B")

mess

Snapshot: 2026-11-04T14:00:00Z · Data license Apache-2.0 · JSON