llamastash /llamastash

Zero-overhead, terminal-native local-LLM orchestrator. Run any engine, any model via the same proxy.

★ 300⑂ 23👁 0⑃ 108● 25▤ 197
RustMITCreated 2026-05-20llamastash.dev
Download: v0.11.0v0.11.0 ↓0.11.02026-11-03
+0
Daily
300
stars
23
forks
0
watchers
108
prs
25
issues
197
releases

README

LlamaStash

Zero-overhead, terminal-native local-LLM orchestrator. Run any engine, any model via the same proxy.


  


!ci
!release




!License
!Code size





A fast TUI and CLI with init wizard for running local LLMs. One Rust binary that's a TUI, a CLI, a daemon, and an OpenAI-compatible proxy. Bring any model and any engine: llama.cpp for GGUFs with zero overhead vs raw llama-server, vLLM and SGLang for safetensors, and a generic backend that runs any other OpenAI-compatible server you declare in config, such as Lemonade and FastFlowLM on the NPU, ds4 for DeepSeek-V4, gufo, Halogen and whisper.cpp. You don't learn each engine's flags, ports and quirks: LlamaStash manages them all the same way, behind one proxy URL and one TUI, CLI and agent surface. See Engines.

!TUI Gif

!TUI

 Contents

- Why
- Install
- Quickstart
- Engines
- Agent Skills
- Features
- Benchmarks
- Screenshots
- Configuration
- CLI exit codes
- Platforms
- Roadmap
- Contributing

 Why

Heavy abstractions (Ollama, LM Studio) hide the engine and tie you to the engines they bundle; running llama-server, vllm serve or any other engine by hand is tedious. LlamaStash is a fast, transparent orchestrator that runs whichever engine fits each model, and it is also a first-class shell-tool surface for agents — one binary, daemon on demand, same primitives in the TUI and the CLI.

 AI agents installing this for a user: jump to INSTALL.md § For AI agents. The non-interactive install + verify contract, and exit-code branching live there.

 Install

Pick whichever channel you prefer — all install the same binary. Full per-platform notes, troubleshooting, and the agent-friendly non-interactive path live in INSTALL.md.

bash
 macOS + Linux, one-shot
curl -fsSL https://llamastash.dev/install.sh | sh

 Windows 11 (PowerShell, no admin elevation)
irm https://llamastash.dev/install.ps1 | iex

 Homebrew (macOS + Linuxbrew)
brew install llamastash/llamastash/llamastash

 Arch Linux (AUR — pick one)
yay -S llamastash        source build

 From crates.io (any platform with a Rust toolchain)
cargo install llamastash

 Windows via Scoop bucket
scoop bucket add llamastash https://github.com/llamastash/scoop-llamastash && scoop install llamastash


_Windows: 64-bit Windows 10 1809+ / Windows 11, PowerShell 5.1+, Windows Terminal recommended for the TUI; the bundled llama-server needs the VC++ 2015–2022 Redistributable (x64). See INSTALL.md._

Then run llamastash init — the interactive wizard installs llama-server for your hardware, downloads a starter GGUF, writes a tuned config, and smoke-launches it.

 Quickstart

bash
 Open the TUI. Scans default caches; daemon auto-spawns on demand.
llamastash

 List discovered models. TTY → padded + table; piped or
 --no-colors → TSV bytes. --json is the agent contract.
llamastash list
llamastash list --json | jq

 Launch a model by name, name substring, path, or canonical id.
llamastash start qwen-coder --ctx 16384 --reasoning on

 run is an alias for start, a

Top issues

Latest release

v0.11.0 v0.11.0 2026-11-03
LlamaStash v0.11.0

This release adds two CLI verbs, so an agent can manage models without the TUI. llamastash delete removes a model's files through the daemon, with the same plan the TUI's Ctrl+D shows. llamastash bench starts a model once per preset and reports load time, prompt and generation speed, and time to first token. Unified-memory hosts also get a configurable admission reserve, and the HuggingFace pull dialog now opens on the models ranked for your hardware.

 Delete a model from the CLI

An agent could pull a model but had no way to remove one. Now it can:

bash
llamastash delete

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON