Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

1-Click LLM Server on Your Phone — no Termux needed! 无需Termux，一键让你的手机变成LLM服务器！

A lightweight chat terminal-interface for llama.cpp server written in C++ with many features and windows/linux support.

Lightweight proxy for LLM

Run Pi coding agent isolated in a Docker Sandbox microVM with a local llama-server as the inference backend

Local LLM proxy, DevOps friendly

CLI wrapper for llama.cpp providing an ollama-like experience

大模型推理服务性能调优

A robust, production-ready Python toolkit to automate the synchronization between a directory of .gguf model files and a llama-swap config.yaml

FastAPI proxy that strips volatile fields from OpenClaw requests to dramatically improve llama-server KV cache hit rates (~22× faster prompt eval)

One place to store and manage all your recipe for Llama Server

Offline-First Local AI Desktop & Mobile Agent.

minimal CLI client for llama-server

A lightweight desktop GUI for llama-server — multi-model routing, per-model presets, live I/O recording. Built with Go. Support Windows · macOS · Linux

Benchmark Gemma 4 E2B on Apple Silicon: MLX (mlx-lm) vs GGUF (llama-server), with TTFT, tokens/sec, and memory.

A production-grade Python SDK for llama-server that streamlines authentication, token rotation, observability, and PII masking—helping AI architects ship secure, traceable LLM systems with enterprise-ready guardrails.

LlamaOrch is simple Bash-based CLI Orchestrator for llama.cpp server.

Enterprise-grade local RAG API assistant backend running on traditional Azure CPU infrastructure. Serves a quantized Llama 3.2 GGUF model via llama-server with offline ChromaDB ingestion and stateful MySQL transaction logging.

Claude-Code-style CLI for your own local LLM fleet. Multi-agent delegation, MCP, hooks, autopilot, 100+ tools (file/shell/web/security/free APIs) — all running on your GPU, no cloud calls.

Terminal UI for launching llama-server/llama.cpp with auto-discovered local GGUF models