Skip to content
View ptimizeroracle's full-sized avatar
🎯
Focusing
🎯
Focusing

Highlights

  • Pro

Block or report ptimizeroracle

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
ptimizeroracle/README.md

Atik

I built data platforms, now LLM platforms: gateways, inference routing, serving on Kubernetes, agents on top.

atiklhy.fr · writing · linkedin · email

Mine

  • Ondine: batch LLM processing for tabular data. Checkpointing, budgets, 100+ providers.

Merged upstream

Reported, fixed upstream

Bugs I reported with a reproduction, then fixed by the maintainers: Polars (2), DuckDB, SGLang, Microsoft Agent Framework (3), NVIDIA NeMo-Relay, vLLM Semantic Router (3), AIBrix.

Recent

  • llm-d-router#1598, measured: bursty traffic piles onto one pod. On 8 simulated pods I compared 8 pickers under bursts and steady prefix-heavy load. No picker wins both: max-score sends 59 of a 128-request burst to one pod (even is 16), while random pickers spread bursts but copy each prefix to 6-7 pods and pay a cold prefill at p95 (298 ms against 44). Bench and raw data.
  • vllm#65569, measured: a proposal routes Linear layers to cuBLAS in vLLM's batch-invariant mode, on the claim that cuBLAS is already invariant there. True on every shape I tried but one: the gpt-oss-20b router (32 x 2880) changes bits with batch size, from 29 rows in bf16 and 2 in fp16. Tested on an H200 across every batch size from 1 to 1024; script and raw data.
  • llm-d-router#3027, merged: two of the three paths that run metrics extractors had no panic recovery. One buggy extractor could kill the router process on the endpoint path; on the Kubernetes path the panic was swallowed and never counted. Both now recover, count the failure in datalayer_extract_errors_total and keep going, like the HTTP path already did.
  • llm-d-router#3379, merged: a pod that had never served a request got no traffic, so it never reported, so it stayed idle. Reproduced on a bench, then confirmed by the reporter on 4 x H100: idle pod in 10 of 12 runs before, 0 of 12 after.
  • aibrix#3297, fixed: least-latency routing scored a pod by its lifetime mean, so a pod that slowed down kept winning for a time that grows with its uptime. I measured the lag and derived when the choice flips; the fix now scores the last minute.

I also review others' PRs, usually with a reproduction or a measurement.

Mostly

  • Data: where I come from. Spark, Kafka, Databricks, dbt, Polars, and the Hadoop era before that.
  • Inference: vLLM, SGLang, LiteLLM, Kubernetes. Queueing theory when the numbers stop making sense.
  • Agents: LangGraph, MCP, evals and tracing. Also Vauban, a reference LLM platform and agent lab I am building as a course: 65 controls, every claim tagged measured or gap. Private for now.

Python, SQL, Go.

Pinned Loading

  1. ondine ondine Public

    Batch-process tabular data with any LLM. pandas & Polars in, structured columns out. Checkpointing, budget limits, and 100+ providers built in.

    Python 5 2

  2. vllm-project/vllm vllm-project/vllm Public

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Python 97.1k 24.9k

  3. vllm-project/semantic-router vllm-project/semantic-router Public

    An open, programmable decision layer for models and compute.

    Go 6.5k 1.1k

  4. vllm-project/production-stack vllm-project/production-stack Public

    vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

    Python 2.7k 562

  5. vllm-project/tpu-inference vllm-project/tpu-inference Public

    Moving to vllm-project/vllm-jax. TPU inference for vLLM, with unified JAX and PyTorch support.

    Python 481 360