File tree Expand file tree Collapse file tree
keps/sig-api-machinery/5647-stale-controller-handling Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -244,6 +244,20 @@ occurred. This will have the same exponential backoff semantics so after a few
244244reconciles of being unable to catch up the requeue will take longer and longer
245245until the cache has enough time to actually catch up to the writes.
246246
247+ ### Circuit Breaking Pattern in Controllers
248+
249+ With the same pattern, we can implement a circuit breaking approach to certain
250+ controllers, such as the node-lifecycle controller. There are scenarios, such
251+ as the obtaining of node lease objects, where controllers may pull stale
252+ information from the cache, causing the controller to wrongly believe that a
253+ lease is expired.
254+
255+ We can perform live gets on these resources to ensure that we don't overreact
256+ to staleness in the cache. Simultaneously if it is determined that the cache
257+ is stale, we can use the same pattern where we mark the cache as not ready
258+ until the cache has at least caught up to our prior live get. This will prevent
259+ too many requests hitting the api server.
260+
247261### Test Plan
248262
249263<!--
You can’t perform that action at this time.
0 commit comments