I built data platforms, now LLM platforms: gateways, inference routing, serving on Kubernetes, agents on top.
atiklhy.fr · writing · linkedin · email
- Ondine: batch LLM processing for tabular data. Checkpointing, budgets, 100+ providers.
- llm-d-router (9) and llm-d (4): inference scheduling on Kubernetes
- AIBrix (2): LLM gateway
- pandas, Narwhals (4), ibis, Great Expectations (2), OpenLineage: dataframes, data quality, lineage
- Cognee: agent memory
Bugs I reported with a reproduction, then fixed by the maintainers: Polars (2), DuckDB, SGLang, Microsoft Agent Framework (3), NVIDIA NeMo-Relay, vLLM Semantic Router (3), AIBrix.
- llm-d-router#1598, measured: bursty traffic piles onto one pod. On 8 simulated pods I compared 8 pickers under bursts and steady prefix-heavy load. No picker wins both: max-score sends 59 of a 128-request burst to one pod (even is 16), while random pickers spread bursts but copy each prefix to 6-7 pods and pay a cold prefill at p95 (298 ms against 44). Bench and raw data.
- vllm#65569, measured: a proposal routes Linear layers to cuBLAS in vLLM's batch-invariant mode, on the claim that cuBLAS is already invariant there. True on every shape I tried but one: the gpt-oss-20b router (32 x 2880) changes bits with batch size, from 29 rows in bf16 and 2 in fp16. Tested on an H200 across every batch size from 1 to 1024; script and raw data.
- llm-d-router#3027, merged: two of the three paths that run metrics extractors had no panic recovery. One buggy extractor could kill the router process on the endpoint path; on the Kubernetes path the panic was swallowed and never counted. Both now recover, count the failure in
datalayer_extract_errors_totaland keep going, like the HTTP path already did. - llm-d-router#3379, merged: a pod that had never served a request got no traffic, so it never reported, so it stayed idle. Reproduced on a bench, then confirmed by the reporter on 4 x H100: idle pod in 10 of 12 runs before, 0 of 12 after.
- aibrix#3297, fixed: least-latency routing scored a pod by its lifetime mean, so a pod that slowed down kept winning for a time that grows with its uptime. I measured the lag and derived when the choice flips; the fix now scores the last minute.
I also review others' PRs, usually with a reproduction or a measurement.
- Data: where I come from. Spark, Kafka, Databricks, dbt, Polars, and the Hadoop era before that.
- Inference: vLLM, SGLang, LiteLLM, Kubernetes. Queueing theory when the numbers stop making sense.
- Agents: LangGraph, MCP, evals and tracing. Also Vauban, a reference LLM platform and agent lab I am building as a course: 65 controls, every claim tagged measured or gap. Private for now.
Python, SQL, Go.




