Skip to content

perf(cli): add in-memory caching for redundant TDP reads#2902

Merged
heyitsaamir merged 6 commits into
mainfrom
heyitsaamir-cli-caching-audit
Jun 24, 2026
Merged

perf(cli): add in-memory caching for redundant TDP reads#2902
heyitsaamir merged 6 commits into
mainfrom
heyitsaamir-cli-caching-audit

Conversation

@heyitsaamir

@heyitsaamir heyitsaamir commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds process-scoped in-memory caching to eliminate redundant TDP (Teams Developer Portal) network reads within a single teams CLI invocation. Started with app-detail reads (the worst hotspot) and extended to bot reads, with a small reusable caching layer shared by both.

Caches are process-scoped (one CLI invocation = one process), Map-backed, and clone-on-read/write (structuredClone) so callers can't corrupt cached objects. Writes refresh the cache from authoritative POST responses; mutations that bypass the normal path explicitly invalidate.

What changed

  • Reusable layersrc/utils/resource-cache.ts: generic createResourceCache<T>() with clone-safe get/set/invalidate.
  • App details (hotspot #1) — fetchAppDetailsV2 is now cache-first with a { force } opt-out; updateAppDetails write-refreshes from its POST response (so uploadIcon benefits); importAppPackage invalidates. This collapses the ~7–9 GET /appdefinitions/v2/{id} reads per teams app update (read-modify-write) down to 1.
  • Bot reads (hotspot #2) — fetchBot and getBotLocation now share a single GET /botframework/{botId} round-trip via one readBotResource() classifier (200 → Teams-managed with details, 404 → Azure). Only those two meaningful outcomes are cached; other statuses throw uncached so transient failures still retry. updateBot refreshes; deleteBot/registerBot invalidate.
  • SpinnerfetchAppDetail keeps its spinner silent when both underlying reads are already warm (fixes the duplicate spinner seen on App → details → back → details).

Tests

  • Unit tests written first (TDD, red→green): app-details-cache.test.ts, bot-cache.test.ts.
  • Gated live timing integration test (RUN_INTEGRATION=1) demonstrating the saved round-trips.
  • Full suite: 206 passed / 34 skipped, tsc --noEmit and npm run build clean.

Notes

  • No TTL — caches live only for the duration of a single command, so staleness across invocations is impossible by construction.
  • Within an invocation, every mutation point (update.ts, doctor.ts, bot/migrate.ts, bot-handler.ts) was traced for read-after-write staleness; all refresh or invalidate correctly.

Add process-scoped in-memory caches to eliminate redundant TDP network
reads within a single CLI invocation:

- Generic createResourceCache<T>() (clone-on-read/write) as a reusable layer
- App-details cache: fetchAppDetailsV2 is cache-first with a {force} opt-out;
  updateAppDetails write-refreshes from its POST response; importAppPackage
  invalidates. Collapses ~7-9 reads per `app update` to 1.
- Bot cache: fetchBot and getBotLocation share one GET /botframework/{botId}
  via a single readBotResource() classifier (200=tm, 404=azure); mutations
  refresh/invalidate. Removes the duplicate spinner/round-trip on app detail.
- Gate the fetchAppDetail spinner to stay silent when both reads are warm.

Includes unit tests (TDD) and a gated live timing integration test
(RUN_INTEGRATION=1).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 24, 2026 00:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds process-scoped, in-memory caching for high-frequency TDP read paths in the teams CLI (app details + bot reads), using a small reusable cache helper to reduce redundant network round-trips within a single invocation.

Changes:

  • Introduce a generic createResourceCache<T>() utility (Map-backed, clone-safe get/set/invalidate).
  • Cache fetchAppDetailsV2 reads with a { force } bypass; refresh cache on updateAppDetails; invalidate on package import.
  • Cache bot read outcomes (200→Teams-managed w/details, 404→Azure), shared by fetchBot and getBotLocation; add tests + an opt-in live integration test.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
packages/cli/src/utils/resource-cache.ts Adds reusable clone-safe, process-scoped cache primitive.
packages/cli/src/apps/app-details-cache.ts App-details cache wrapper + invalidation API.
packages/cli/src/apps/bot-cache.ts Bot read-result cache wrapper shared across call sites.
packages/cli/src/apps/api.ts Makes fetchAppDetailsV2 cache-first w/ { force }; refreshes cache on update.
packages/cli/src/apps/tdp.ts Adds cached bot reader + bot cache refresh/invalidate hooks; invalidates app-details cache on import.
packages/cli/src/apps/bot-location.ts Reuses cached bot reader instead of issuing its own GET.
packages/cli/src/apps/home.ts Avoids spinner noise when required resources are already warm in cache.
packages/cli/src/apps/index.ts Re-exports cache modules.
packages/cli/tests/app-details-cache.test.ts Unit tests for app-details caching/force/invalidate/clone-safety.
packages/cli/tests/bot-cache.test.ts Unit tests for bot caching, shared read, refresh, invalidate, clone-safety.
packages/cli/tests/app-details-cache.integration.test.ts Opt-in live test demonstrating reduced round-trips.
packages/cli/tests/update-app-details-bump.test.ts Ensures cache is cleared between tests to avoid cross-test coupling.

Comment thread packages/cli/src/apps/tdp.ts
Comment thread packages/cli/tests/app-details-cache.test.ts Outdated
Comment thread packages/cli/tests/app-details-cache.integration.test.ts Outdated
heyitsaamir and others added 2 commits June 23, 2026 17:24
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Comment thread packages/cli/src/apps/api.ts Outdated
Comment thread packages/cli/src/utils/resource-cache.ts
heyitsaamir and others added 2 commits June 24, 2026 10:04
Can we also update our linting rules to include stuff like this?

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
use == null instead

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
… noUnusedLocals lint rule

- Fix extra `}` in resource-cache.ts invalidate method introduced by previous autofix commit
- Remove unused imports: AppBot (api.ts), fetchApp (bot/get.ts), createTdpBotHandler (update.ts), select and logger (az-prompts.ts)
- Add noUnusedLocals: true to tsconfig.json so tsc --noEmit catches unused imports going forward
@heyitsaamir heyitsaamir merged commit 1f11cda into main Jun 24, 2026
4 checks passed
@heyitsaamir heyitsaamir deleted the heyitsaamir-cli-caching-audit branch June 24, 2026 21:13
heyitsaamir added a commit that referenced this pull request Jul 2, 2026
## Summary

Prepare stable CLI 3.0.2 by merging current `main` into `release/v3` and
setting explicit stable release metadata.

## CLI package commits since 3.0.1

- `1f11cdaf1` perf(cli): add in-memory caching for redundant TDP reads
(#2902)
- `2a969dd29` Fix CLI template issues and remove AI/MCP/MCPClient
templates (#2904)
- `553685c6b` Confirm interactive RSC permission updates (#2906)
- `082e79830` Merge main into release/v3 for CLI 3.0.2
- `17d2170a6` Prepare CLI 3.0.2 release

## Validation

- `npx turbo build --filter=@microsoft/teams.cli`
- `npm -w @microsoft/teams.cli test`

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Corina <14900841+corinagum@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Umang <3513186+umangsehgal@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: SirajShaik-MSFT <v-sirshaikh@microsoft.com>
Co-authored-by: Hugo Gonzalez <hugogonzalez@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Kavin <115390646+singhk97@users.noreply.github.com>
Co-authored-by: Rido <rido-min@users.noreply.github.com>
Co-authored-by: Ricky Castaneda <126518272+MSFTRickyCastaneda@users.noreply.github.com>
Co-authored-by: Shanmathi Mayuram Krithivasan <37715033+ShanmathiMayuramKrithivasan@users.noreply.github.com>
Co-authored-by: Nick Walker <nickwalk@microsoft.com>
Co-authored-by: Mehak Bindra <mhk0397@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants