+0
Daily
300
stars
23
forks
0
watchers
108
prs
25
issues
197
releases
README
LlamaStash Zero-overhead, terminal-native local-LLM orchestrator. Run any engine, any model via the same proxy. !ci !release !License !Code size A fast TUI and CLI with init wizard for running local LLMs. One Rust binary that's a TUI, a CLI, a daemon, and an OpenAI-compatible proxy. Bring any model and any engine: llama.cpp for GGUFs with zero overhead vs raw llama-server, vLLM and SGLang for safetensors, and a generic backend that runs any other OpenAI-compatible server you declare in config, such as Lemonade and FastFlowLM on the NPU, ds4 for DeepSeek-V4, gufo, Halogen and whisper.cpp. You don't learn each engine's flags, ports and quirks: LlamaStash manages them all the same way, behind one proxy URL and one TUI, CLI and agent surface. See Engines. !TUI Gif !TUI Contents - Why - Install - Quickstart - Engines - Agent Skills - Features - Benchmarks - Screenshots - Configuration - CLI exit codes - Platforms - Roadmap - Contributing Why Heavy abstractions (Ollama, LM Studio) hide the engine and tie you to the engines they bundle; running llama-server, vllm serve or any other engine by hand is tedious. LlamaStash is a fast, transparent orchestrator that runs whichever engine fits each model, and it is also a first-class shell-tool surface for agents — one binary, daemon on demand, same primitives in the TUI and the CLI. AI agents installing this for a user: jump to INSTALL.md § For AI agents. The non-interactive install + verify contract, and exit-code branching live there. Install Pick whichever channel you prefer — all install the same binary. Full per-platform notes, troubleshooting, and the agent-friendly non-interactive path live in INSTALL.md. bash macOS + Linux, one-shot curl -fsSL https://llamastash.dev/install.sh | sh Windows 11 (PowerShell, no admin elevation) irm https://llamastash.dev/install.ps1 | iex Homebrew (macOS + Linuxbrew) brew install llamastash/llamastash/llamastash Arch Linux (AUR — pick one) yay -S llamastash source build From crates.io (any platform with a Rust toolchain) cargo install llamastash Windows via Scoop bucket scoop bucket add llamastash https://github.com/llamastash/scoop-llamastash && scoop install llamastash _Windows: 64-bit Windows 10 1809+ / Windows 11, PowerShell 5.1+, Windows Terminal recommended for the TUI; the bundled llama-server needs the VC++ 2015–2022 Redistributable (x64). See INSTALL.md._ Then run llamastash init — the interactive wizard installs llama-server for your hardware, downloads a starter GGUF, writes a tuned config, and smoke-launches it. Quickstart bash Open the TUI. Scans default caches; daemon auto-spawns on demand. llamastash List discovered models. TTY → padded + table; piped or --no-colors → TSV bytes. --json is the agent contract. llamastash list llamastash list --json | jq Launch a model by name, name substring, path, or canonical id. llamastash start qwen-coder --ctx 16384 --reasoning on run is an alias for start, a
Top issues
Latest release
v0.11.0 v0.11.0 2026-11-03LlamaStash v0.11.0 This release adds two CLI verbs, so an agent can manage models without the TUI. llamastash delete removes a model's files through the daemon, with the same plan the TUI's Ctrl+D shows. llamastash bench starts a model once per preset and reports load time, prompt and generation speed, and time to first token. Unified-memory hosts also get a configurable admission reserve, and the HuggingFace pull dialog now opens on the models ranked for your hardware. Delete a model from the CLI An agent could pull a model but had no way to remove one. Now it can: bash llamastash delete