Skip to content

Commit dffa45e

Browse files
committed
Add section on circuit breaking behavior
1 parent 389b87a commit dffa45e

1 file changed

Lines changed: 14 additions & 0 deletions

File tree

  • keps/sig-api-machinery/5647-stale-controller-handling

keps/sig-api-machinery/5647-stale-controller-handling/README.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,20 @@ occurred. This will have the same exponential backoff semantics so after a few
244244
reconciles of being unable to catch up the requeue will take longer and longer
245245
until the cache has enough time to actually catch up to the writes.
246246

247+
### Circuit Breaking Pattern in Controllers
248+
249+
With the same pattern, we can implement a circuit breaking approach to certain
250+
controllers, such as the node-lifecycle controller. There are scenarios, such
251+
as the obtaining of node lease objects, where controllers may pull stale
252+
information from the cache, causing the controller to wrongly believe that a
253+
lease is expired.
254+
255+
We can perform live gets on these resources to ensure that we don't overreact
256+
to staleness in the cache. Simultaneously if it is determined that the cache
257+
is stale, we can use the same pattern where we mark the cache as not ready
258+
until the cache has at least caught up to our prior live get. This will prevent
259+
too many requests hitting the api server.
260+
247261
### Test Plan
248262

249263
<!--

0 commit comments

Comments
 (0)