Raj Firke

Software Engineer (AI) at Red Hat · Researcher (LLM evaluation & CoT process monitoring) · 2x EMNLP Main · Open Source

I am a Software Engineer (AI) at Red Hat. I work on agentic systems in production: MCP servers, diagnostics agents, and support automation that customers actually use.

I work on LLM evaluation and chain-of-thought monitoring. I care about when a score or a trace stops being a trustworthy signal. I look at whether two models can share the same accuracy and still fail in different ways, and whether a chain that still looks clean is already getting riskier with depth.

I belong in the overlap of shipping agents and measuring them. I maintain Provena. I contribute to PyTorch and vLLM. I mentor students who are starting the same kind of work.

Patents

• System and Method for Utilizing Digital Footprints of a User to Generate Conversational AI Thereof 202421048008 · Filed Jun 2024 · Granted (20-year term)

• Recommendation and Intent Reconciliation in a Virtual Leader Framework 202521025963 · Filed Apr 2025 · Under review

Publications

• When Does Reasoning Age? Survival Analysis of Step-Level Error Hazard in LLM Chains · EMNLP 2026 Main · first author · CoT process monitoring

• The Correlation Mirage: Benchmark Dependence Collapses for Top-Performing LLMs · EMNLP 2026 Main · first author · science of evaluations

• GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon · ICCUBEA 2026 (IEEE Xplore) · energy as an eval axis

• A Survey on Advanced Recommendation Systems: Content-Based Filtering, Collaborative Filtering, Hybrid and Opinion Mining Approaches · ICTIS 2025 / Springer LNNS · 2025

• Proposed Model of Hindi Book Review Sentiment Analysis · IJERT 2023 · earlier NLP

Active contributor working on core framework improvements:

• Input validation & safety — Adding proper bounds checking to prevent silent failures in max_pool3d, channel_shuffle, RNN cells, and convolution ops
• Optimizer improvements — maximize parameter for LBFGS, integer step tensor support in foreach optimizers
• Numerical stability — Fixing NaN propagation in lp_pool, hardtanh backward pass corrections
• API enhancements — keepdim for cosine_similarity, NanDetectMode for forward-pass diagnostics, dtype context manager

Contributing to the high-throughput LLM serving engine:

• Responses API — Namespace tools support for harmony/GPT-OSS models

Languages

ML & AI

Infrastructure & DevOps

Databases & Tools