Requesting retry logic when model is overloaded

[sethladd]

I'm seeing:

 [WARN] 5 concept(s) planned but only 4 written for foo (ServiceUnavailableError).                        
      summary-rewrite.................openkb.agent.compiler WARNING: summary-rewrite failed for foo:           
  litellm.ServiceUnavailableError: GeminiException - {                                                                                          
    "error": {                                                                                                                                  
      "code": 503,                                                                                                                              
      "message": "This model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later.",           
      "status": "UNAVAILABLE"                                                                                                                   
    }                                                                                                                                           
  }

Which apparently looks like I lost information? Not sure if there's a retry, but it looks like a retry isn't happening?

Hope this helps!

[KylinMountain]

You can get this today without waiting on a release — OpenKB already forwards litellm's own retry setting through the litellm: passthrough block. In .openkb/config.yaml:

litellm:
  num_retries: 3

litellm uses num_retries as the default retry count for every LLM call (it's applied as a module-level global), so transient failures — Gemini 503, Anthropic 529 overloaded, etc. — get retried automatically across compile, query, and chat, with exponential backoff.