feat: add Phase 1 Python client package over the server

.github/workflows/python.yml

@@ -0,0 +1,50 @@
+# PR / push CI for the Python client package under python/. Runs ruff (lint +
+# format), mypy (type check), and pytest (unit + lifecycle tests). It needs
+# neither MLX nor the mlxcel binary, so it runs on plain ubuntu-latest and is
+# independent of the Rust CI in ci.yml. The e2e test is skipped here because it
+# requires a built binary (set MLXCEL_BIN to run it locally).
+
+name: Python
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'python/**'
+      - '.github/workflows/python.yml'
+  pull_request:
+    branches: [main]
+    paths:
+      - 'python/**'
+      - '.github/workflows/python.yml'
+
+permissions:
+  contents: read
+
+jobs:
+  check:
+    name: lint, type-check, test
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Install package with dev extras
+        run: python -m pip install --upgrade pip && pip install -e "python[dev]"
+
+      - name: Ruff lint
+        run: ruff check python
+
+      - name: Ruff format check
+        run: ruff format --check python
+
+      - name: Mypy
+        run: mypy python/src
+
+      - name: Pytest (unit + lifecycle)
+        run: pytest python/tests -m "not e2e" -q

README.md

@@ -265,6 +265,21 @@ mlxcel arch
 
 for the CLI summary, and see [Supported models](docs/supported-models.md) for the maintained architecture table, known limitations, and VLM coverage notes.
 
+## Python
+
+`mlxcel` ships a pure-Python client that drives the OpenAI-compatible server from Python. It spawns and manages a local `mlxcel serve` process (managed mode) or connects to a running one (connect mode), auto-discovers the served model id, and exposes the raw `openai` client for the full API surface.
+
+```python
+import mlxcel
+
+with mlxcel.LLM("mlx-community/Qwen3-4B-4bit") as llm:
+    print(llm.generate("def fib(n):", max_tokens=128))
+    for delta in llm.stream("Write a haiku about autumn"):
+        print(delta, end="", flush=True)
+```
+
+Install with `pip install ./python`. See [Python client](docs/python-client.md) for managed and connect modes, streaming, structured output, async usage, and troubleshooting. The client lives in [`python/`](python) and builds entirely on the existing server (no native extension).
+
 ## Optional GUI
 
 `mlxcel-server` can be used directly through HTTP clients. For a local graphical front-end, [Backend.AI Go](https://go.backend.ai) can be used as a companion UI for chat, model management, and multi-model routing.
@@ -279,6 +294,7 @@ for the CLI summary, and see [Supported models](docs/supported-models.md) for th
 - [Tensor and pipeline parallelism](docs/distributed.md)
 - [TurboQuant KV cache](docs/turbo-kv-cache.md)
 - [OpenAI Responses API](docs/responses-api.md)
+- [Python client](docs/python-client.md)
 - [Adding a new model](docs/adding-models.md)
 
 ## Contributing

docs/README.md

@@ -25,6 +25,7 @@ Current GitHub-facing docs:
 10. `audio-api.md` — implemented `/v1/audio` endpoints: Whisper STT setup, request/response reference, WAV encoding details, and request validation order.
 11. `adding-models.md` — contribution guide for new model architectures.
 12. `block-diffusion.md` — DiffusionGemma block-diffusion generation: canvas denoising vs autoregressive, CLI flags, throughput, and phase 1 limitations.
+13. `python-client.md`: the `mlxcel` Python client over the OpenAI-compatible server, covering managed and connect modes, streaming, chat, structured output, the `openai_client` escape hatch, async usage, and troubleshooting.
 
 ## Architecture Decision Records
 

docs/python-client.md

@@ -0,0 +1,198 @@
+# Python client
+
+`mlxcel` ships a pure-Python client that drives the OpenAI-compatible server (`mlxcel serve`) from Python. It either spawns and supervises a local server process (managed mode) or connects to an already-running one (connect mode), auto-discovers the served model id, and exposes the raw `openai` client for the full API surface (tools, `response_format`, logprobs, multimodal).
+
+This is Phase 1 of Python integration. It builds entirely on the existing HTTP server: no native extension, and no changes to the Rust inference core. The package lives in [`python/`](../python) and is published under the import name `mlxcel`.
+
+## Install
+
+```bash
+pip install ./python          # from a repo checkout
+pip install ./python[dev]     # adds pytest, ruff, mypy
+```
+
+Requires Python 3.9 or newer. Runtime dependencies are `openai>=1.40` and `httpx>=0.27`. Managed mode additionally needs the `mlxcel` binary; the client finds it via the `binary=` argument, the `MLXCEL_BIN` environment variable, or `mlxcel` on `PATH`, in that order. See [Installation](installation.md) for building the binary.
+
+## Managed mode
+
+Pass a model and the client spawns `mlxcel serve`, waits until the server reports ready, and stops it on exit. The model loads and warms up before the listener binds, so a first run that downloads weights can take a while; the client polls `/health` with a generous `startup_timeout` (default 600 seconds) and watches the child process for an early exit.
+
+```python
+import mlxcel
+
+with mlxcel.LLM("mlx-community/Qwen3-4B-4bit") as llm:
+    text = llm.generate("def fib(n):", max_tokens=128, temperature=0.7)
+    print(text)
+
+    reply = llm.chat([{"role": "user", "content": "Hello"}], max_tokens=64)
+    print(reply)
+
+    print(llm.model)       # resolved model id, auto-discovered from /v1/models
+    print(llm.models())    # ["<id>"]
+    ids = llm.tokenize("hello world")
+    print(llm.detokenize(ids))
+```
+
+The context manager (`with`) is the recommended form; it guarantees shutdown. The client also registers an `atexit` cleanup and a best-effort finalizer so a leaked handle still stops the server.
+
+Useful managed-mode arguments: `binary=`, `host=`/`port=` (forces TCP and binds there), `socket=` (an explicit Unix socket path), `api_key=`, `ctx_size=`, `n_predict=`, `alias=`, `warmup=`, `extra_args=[...]` (forwarded verbatim to `mlxcel serve`), and `startup_timeout=`.
+
+## Connect mode
+
+Pass `base_url=` or `socket=` (but not a model) to talk to a server you started yourself. No subprocess is launched or managed.
+
+```python
+# TCP
+llm = mlxcel.LLM(base_url="http://localhost:8080/v1")
+
+# Unix domain socket
+llm = mlxcel.LLM(socket="/tmp/mlxcel.sock")
+
+print(llm.generate("Hello"))
+llm.close()
+```
+
+Start a matching server with the CLI:
+
+```bash
+# TCP
+mlxcel serve -m mlx-community/Qwen3-4B-4bit --host 127.0.0.1 --port 8080
+
+# Unix domain socket: --port 0 reinterprets --host as the socket path
+mlxcel serve -m mlx-community/Qwen3-4B-4bit --host /tmp/mlxcel.sock --port 0
+```
+
+Passing both a model and a connect target raises `MlxcelError`, because the mode would be ambiguous.
+
+## Security: multi-user hosts
+
+On a shared machine, the default socket path under `/tmp` is world-readable: any local user can connect to the server you spawned and send requests. If that matters for your deployment, pass an explicit `socket=` path under a directory only you can read, for example one under `$XDG_RUNTIME_DIR` (mode `0700`, owned by your uid):
+
+```python
+import os, pathlib, mlxcel
+
+runtime_dir = pathlib.Path(os.environ.get("XDG_RUNTIME_DIR", f"/run/user/{os.getuid()}"))
+runtime_dir.mkdir(mode=0o700, parents=True, exist_ok=True)
+
+with mlxcel.LLM("mlx-community/Qwen3-4B-4bit", socket=str(runtime_dir / "mlxcel.sock")) as llm:
+    print(llm.generate("hello"))
+```
+
+The CLI equivalent is `mlxcel serve --host "$XDG_RUNTIME_DIR/mlxcel.sock" --port 0`.
+
+On macOS, `$TMPDIR` already expands to a per-user path under `/var/folders`, so the default socket is private there. On Linux without an active login session, `$XDG_RUNTIME_DIR` may be absent; fall back to a `0700` subdirectory under your home directory if needed.
+
+## Streaming
+
+`stream` and `chat_stream` yield text deltas as they arrive.
+
+```python
+with mlxcel.LLM("mlx-community/Qwen3-4B-4bit") as llm:
+    for delta in llm.stream("Write a haiku about autumn"):
+        print(delta, end="", flush=True)
+
+    for delta in llm.chat_stream([{"role": "user", "content": "List three uses for a Pi."}]):
+        print(delta, end="", flush=True)
+```
+
+## Chat
+
+`chat` returns the assistant message content as a string.
+
+```python
+messages = [
+    {"role": "system", "content": "You are concise."},
+    {"role": "user", "content": "What is MLX?"},
+]
+print(llm.chat(messages, max_tokens=128, temperature=0.3))
+```
+
+## Sampling parameters
+
+Generation methods accept OpenAI sampling fields directly: `max_tokens`, `temperature`, `top_p`, `stop`, `seed`, `presence_penalty`, `frequency_penalty`, `logit_bias`, and `response_format`. Server-specific knobs that are not part of the OpenAI schema (`top_k`, `min_p`, `repetition_penalty`, and DRY settings) are forwarded in the request body. You can also pass an explicit `extra_body={...}` for arbitrary server fields; values you set there win on conflict.
+
+```python
+llm.generate("Once upon a time", max_tokens=200, top_p=0.9, top_k=40, min_p=0.05)
+llm.generate("...", extra_body={"repetition_penalty": 1.1})
+```
+
+## Structured output
+
+The server's llguidance-backed constrained decoding honors `response_format`, so you can require schema-valid JSON.
+
+```python
+import json
+
+schema = {
+    "type": "json_schema",
+    "json_schema": {
+        "name": "person",
+        "schema": {
+            "type": "object",
+            "properties": {
+                "name": {"type": "string"},
+                "age": {"type": "integer"},
+            },
+            "required": ["name", "age"],
+            "additionalProperties": False,
+        },
+    },
+}
+
+reply = llm.chat(
+    [{"role": "user", "content": "Invent a person as JSON."}],
+    response_format=schema,
+    max_tokens=128,
+)
+person = json.loads(reply)
+```
+
+## The `openai_client` escape hatch
+
+For anything the convenience methods do not cover (tools, logprobs, vision and audio inputs, the Responses API), reach for the configured OpenAI client directly. It is wired to the same transport, so TCP and Unix-socket setups both work.
+
+```python
+oai = llm.openai_client   # openai.OpenAI (or openai.AsyncOpenAI for AsyncLLM)
+oai.chat.completions.create(
+    model=llm.model,
+    messages=[{"role": "user", "content": "Weather in SF?"}],
+    tools=[...],
+)
+```
+
+## Async usage
+
+`mlxcel.AsyncLLM` mirrors the synchronous API with `async`/`await` and async iterators, backed by `openai.AsyncOpenAI` and `httpx.AsyncClient`.
+
+```python
+import asyncio
+import mlxcel
+
+async def main():
+    async with mlxcel.AsyncLLM("mlx-community/Qwen3-4B-4bit") as llm:
+        print(await llm.generate("def fib(n):", max_tokens=128))
+        async for delta in llm.stream("Write a haiku"):
+            print(delta, end="", flush=True)
+
+asyncio.run(main())
+```
+
+The managed-server lifecycle (spawn and readiness polling) runs synchronously inside the constructor since it is a one-time blocking setup; every generation call is async. The model id resolves lazily on the first request, so read `llm.model` only after a call has run, or pass an explicit `model=` override.
+
+## Errors
+
+| Exception | Raised when |
+|-----------|-------------|
+| `MlxcelError` | base class for all client errors (also used for ambiguous arguments) |
+| `MlxcelServerError` | a managed server fails to launch, become ready, or stay alive (carries the captured stderr tail) |
+| `MlxcelTimeoutError` | the managed server does not become ready within `startup_timeout` |
+
+HTTP and API errors from the server propagate as native `openai` SDK exceptions (for example `openai.APIStatusError`), so status codes and response bodies stay visible. Only lifecycle concerns are wrapped in `Mlxcel*` types.
+
+## Troubleshooting
+
+- **Server never becomes ready.** The first run downloads weights before binding the listener. Increase `startup_timeout`, and enable logging to watch progress: `logging.getLogger("mlxcel.server").setLevel(logging.INFO)`. Server stderr (including download and load progress) is forwarded to that logger.
+- **`MlxcelServerError` on startup with a stderr tail.** The child process exited before becoming ready. The attached stderr usually names the cause (model not found, out of memory, bad flag).
+- **Binary not found.** Pass `binary="/path/to/mlxcel"`, set `MLXCEL_BIN`, or put `mlxcel` on `PATH`.
+- **Unix socket path too long.** `sun_path` is about 104 bytes on macOS and 108 on Linux. On macOS, `$TMPDIR` resolves to a long `/var/folders/...` path, so the client defaults the socket to a short name under `/tmp` instead. If your explicit `socket=` path exceeds the limit, the client raises a clear error; pass a shorter path.
+- **Port already in use (TCP).** In managed mode without an explicit `port=`, the client picks a free ephemeral port. Pass `port=` only when you need a fixed one.

mkdocs.ko.yml

@@ -146,6 +146,7 @@ nav:
       - user-guide/index.md
       - CLI 명령어: user-guide/cli.md
       - 서버 모드: user-guide/server.md
+      - Python 클라이언트: user-guide/python-client.md
       - 비전-언어 모델: user-guide/vision-language-models.md
       - LoRA 어댑터: user-guide/lora.md
       - 추측적 디코딩: user-guide/speculative-decoding.md

mkdocs.yml

@@ -143,6 +143,7 @@ nav:
       - user-guide/index.md
       - CLI Commands: user-guide/cli.md
       - Server Mode: user-guide/server.md
+      - Python Client: user-guide/python-client.md
       - Vision-Language Models: user-guide/vision-language-models.md
       - LoRA Adapters: user-guide/lora.md
       - Speculative Decoding: user-guide/speculative-decoding.md

python/.gitignore

@@ -0,0 +1,10 @@
+# Python build / cache artifacts for the mlxcel client package.
+.venv/
+__pycache__/
+*.py[cod]
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+*.egg-info/
+dist/
+build/

python/README.md

@@ -0,0 +1,80 @@
+# mlxcel (Python client)
+
+A thin, pure-Python client for the [mlxcel](https://github.com/lablup/mlxcel) OpenAI-compatible inference server. It spawns and manages a local `mlxcel serve` process (managed mode) or connects to an already-running one (connect mode), auto-discovers the served model id, and exposes the raw `openai` client for the full API surface.
+
+This is Phase 1 of Python integration: it builds entirely on the existing HTTP server, so it needs no native extension and no changes to the Rust inference core.
+
+## Install
+
+```bash
+pip install ./python          # from a repo checkout
+pip install ./python[dev]     # with pytest, ruff, mypy for development
+```
+
+Requires Python 3.9+. The client itself is pure Python (`openai>=1.40`, `httpx>=0.27`). Managed mode additionally needs the `mlxcel` binary on `PATH`, or pass `binary=` / set `MLXCEL_BIN`.
+
+## Usage
+
+```python
+import mlxcel
+
+# Managed mode: spawn and supervise a local server.
+with mlxcel.LLM("mlx-community/Qwen3-4B-4bit") as llm:
+    print(llm.generate("def fib(n):", max_tokens=128, temperature=0.7))
+
+    for delta in llm.stream("Write a haiku about autumn"):
+        print(delta, end="", flush=True)
+
+    print(llm.chat([{"role": "user", "content": "Hello"}], max_tokens=64))
+
+    print(llm.model)        # resolved model id (auto-discovered)
+    print(llm.models())     # ["<id>"]
+    ids = llm.tokenize("hello world")
+    print(llm.detokenize(ids))
+
+    # Escape hatch: the full OpenAI surface (tools, response_format, logprobs).
+    oai = llm.openai_client
+    oai.chat.completions.create(model=llm.model, messages=[...], tools=[...])
+
+# Connect mode: talk to an already-running server.
+llm = mlxcel.LLM(base_url="http://localhost:8080/v1")  # TCP
+llm = mlxcel.LLM(socket="/tmp/mlxcel.sock")            # Unix socket
+```
+
+Async usage mirrors the sync API via `mlxcel.AsyncLLM` (`await llm.generate(...)`, `async for delta in llm.stream(...)`).
+
+## Modes
+
+- **Managed mode** (default when `model=` is given): the client spawns `mlxcel serve`, waits until `/health` returns ready, forwards server logs to the `mlxcel.server` Python logger, and stops the process on exit.
+- **Connect mode** (when `base_url=` or `socket=` is given): no subprocess; the client just talks to the server.
+- Passing both a model and a connect target raises `MlxcelError`.
+
+On POSIX, managed mode defaults to a Unix domain socket for low-overhead local IPC. Keep socket paths short (`sun_path` is about 104 bytes on macOS, 108 on Linux); the default lives under `/tmp`. Pass `socket=` to override. Windows uses TCP.
+
+## Sampling parameters
+
+`generate`, `stream`, `chat`, and `chat_stream` accept OpenAI sampling fields directly: `max_tokens`, `temperature`, `top_p`, `stop`, `seed`, `presence_penalty`, `frequency_penalty`, `logit_bias`, `response_format`. Server-specific knobs (`top_k`, `min_p`, `repetition_penalty`, DRY settings) are forwarded in the request body; you can also pass an explicit `extra_body={...}`.
+
+## Errors
+
+- `MlxcelError`: base class.
+- `MlxcelServerError`: launch, readiness, or crash failures (carries the server stderr tail).
+- `MlxcelTimeoutError`: readiness timeout.
+
+HTTP and API errors surface as native `openai` SDK exceptions (for example `openai.APIStatusError`), not as `Mlxcel*` types.
+
+## Tests
+
+```bash
+pip install -e ./python[dev]
+ruff check python
+ruff format --check python
+mypy python/src
+pytest python/tests -m "not e2e"      # unit + lifecycle, no binary needed
+```
+
+The end-to-end test (`-m e2e`) is skipped unless `MLXCEL_BIN` points at a built binary:
+
+```bash
+MLXCEL_BIN=/path/to/mlxcel pytest python/tests/test_e2e.py -m e2e
+```