The fastest 本地 AI 引擎为 Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% 工具 calling. 17 工具 parsers, prompt 缓存, reasoning separation, 云端 routing. Drop-in OpenAI replacement. Works 与 Claude 代码, Cursor, Aider.GitHub 3,147 ⭐ / 364 Fork，主要使用 Python 开发。标签：apple-silicon,claude-code,cursor,deep……
项目介绍
The fastest 本地 AI 引擎为 Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% 工具 calling. 17 工具 parsers, prompt 缓存, reasoning separation, 云端 routing. Drop-in OpenAI replacement. Works 与 Claude 代码, Cursor, Aider.
核心功能
- tok/s (tokens per second) — roughly how many words the AI generates per second. Higher = faster. - 4bit / 8bit — compression levels for models. 4bit uses less memory (recommended); 8bit is higher quality. - TTFT (Time To First Token) — how long before the AI starts responding. - Tool calling — the AI can call functions in your code. Used by Cursor, Claude Code, and coding assistants. - OpenAI API compatible — Rapid-MLX speaks the same language as ChatGPT's API, so any app that works with ChatGPT can work with Rapid-MLX by just changing the server address. - Ollama / llama.cpp — other popular tools for running local AI. The only apples-to-apples row in our table is GPT-OSS 20B (identical weights both sides) — Rapid-MLX runs it 2.3x faster than Ollama under B=4 concurrent load. On the Qwen3 closest-tag rows (Qwen3.5/3.6 DeltaNet isn't on llama.cpp yet, so we compare against `qwen3:Nb`) Rapid-MLX leads 1.7–2.4x. The Gemma 4 row is tied at parity with Ollama's Gemma 3 (different architectures, 1.0x). Against `mlx-lm serve` (same MLX weights) Rapid-MLX is 1.2–1.5x faster. Full caveats in [Benchmarks](#benchmarks).
使用指南
<img src="https://raw.githubusercontent.com/raullenchai/Rapid-MLX/main/docs/assets/demo.gif" alt="Rapid-MLX demo — install, serve Gemma 4, chat, tool calling" width="700"> <br> <em>pip install → serve Gemma 4 26B → chat + tool calling → works with PydanticAI, LangChain, Aider, and more.</em> </p> <sub>Single-user end-to-end throughput (B=1: one request at a time, 256 max output tokens, `output_tokens / wall-clock` incl. first-token latency), median of 3 rounds. `chat_template_kwargs.enable_thinking=False` passed where the engine honours it. Tested on M3 Ultra 256 GB / rapid-mlx v0.6.83 (fused top-p sampler). ¹ carried over from 2026-04 bench — disk-constrained on this refresh.</sub> <details> <summary><b>New to local AI? Quick glossary</b></summary> - tok/s (tokens per second) — roughly how many words the AI generates per second. Higher = faster. - 4bit / 8bit — compression levels for models. 4bit uses less memory (recommended); 8bit is higher quality. - TTFT (Time To First Token) — how long before the AI starts responding. - Tool calling — the AI can call functions in your code. Used by Cursor, Claude Code, and coding assistants. - OpenAI API compatible — Rapid-MLX speaks the same language as ChatGPT's API, so any app that works with ChatGPT can work with Rapid-MLX by just changing the server address. </details>
项目信息
| 项目 | 信息 |
|---|---|
| 项目名称 | raullenchai/Rapid-MLX |
| Star 数 | 3,147 |
| Fork 数 | 364 |
| 编程语言 | Python |
| 开源协议 | Apache-2.0 |
| 创建时间 | 2026-02-25 |
| 最近更新 | 2026-06-30 |
| 主题标签 | apple-silicon、claude-code、cursor、deepseek、fastapi |
项目地址
GitHub 仓库：<https://github.com/raullenchai/Rapid-MLX>
> 完整 URL：[https://github.com/raullenchai/Rapid-MLX](https://github.com/raullenchai/Rapid-MLX)
如需查看完整源码、安装命令、使用示例、API 文档，请直接访问上方项目地址。所有信息均以 GitHub 仓库为准。