Caching is not fully optimised in the planner and workflow agents. By optimising prompt ordering and cache breakpoints, we could speed up and lower the costs of the global assistant. Any inefficiency in the workflow agent will become more urgent to fix with the launch of the global assistant.
Implement caching properly and measure Time to First Token and estimate token cost before & after.
Caching is not fully optimised in the planner and workflow agents. By optimising prompt ordering and cache breakpoints, we could speed up and lower the costs of the global assistant. Any inefficiency in the workflow agent will become more urgent to fix with the launch of the global assistant.
Implement caching properly and measure Time to First Token and estimate token cost before & after.