[WIP] Add token-efficiency techniques across AI generation pipeline

[Copilot]

• Create services/tokenEstimator.ts — token estimation utility (chars/4 heuristic, input/output tracking)
• Create services/llmCache.ts — response caching keyed by (model, compacted prompt, schema, temperature) using IndexedDB
• Create services/llmOrchestrator.ts — central prompt compaction, context budgeting, jittered retry, chunking, cache integration
• Update types.ts — add TokenMetrics, ChapterMemory, ProjectTokenStats types; extend Chapter
• Update services/storageService.ts — persist chapter summaries and token metrics
• Refactor services/geminiService.ts — use orchestrator for all calls, add chapter-summary extraction, compact system instructions, low-token critique pass
• Create components/shared/TokenMetricsDisplay.tsx — surface per-step token usage and cache hit rate
• Update App.tsx — wire in token metrics state and display widget

Original prompt

Goal: Incorporate token-efficiency techniques across the AI generation pipeline in crazyrob425/KDP-E-Book-Generator without reducing output quality.

Implement a comprehensive token-optimization layer and refactor existing Gemini calls to use it. Focus on reducing prompt size, eliminating redundant context, caching, structured outputs, incremental diff-based rewrites, and multi-pass orchestration that minimizes re-sending long drafts.

Requirements:

1. Add a central prompt/LLM orchestration utility that provides:

   • Prompt compaction (whitespace normalization, list compression, removal of duplicate instructions)
   • Context window budgeting with hard limits per request (estimate tokens by chars heuristic)
   • Deterministic JSON schema outputs when possible
   • Automatic chunking for long chapter generation and rewrites
   • Retry with jittered exponential backoff (reuse existing behavior)
   • Response caching keyed by (model, compacted prompt, schema, temperature) to avoid repeat calls.

2. Add a "memory" mechanism to avoid re-sending full manuscripts:

   • Create a rolling chapter summary + key facts extraction step per chapter.
   • Store summaries in IndexedDB alongside chapters.
   • For rewrite/regenerate operations, send only: (chapter summary + specific excerpt needing edits + editor directives) instead of full chapter.

3. Refactor services/geminiService.ts to:

   • Use the orchestration utility for all calls.
   • Add consistent system instruction templates shared across agents.
   • Switch to schema-first outputs where feasible.
   • Implement a low-token "critique pass" producing compact bullet directives.

4. Add UI/telemetry for token savings:

   • Track estimated input/output tokens per call (approx) and aggregate per project.
   • Display per-step token usage and cache hit rate.

5. Ensure zero-budget constraint:

   • No paid 3rd-party services.
   • Use only local caching/persistence.

6. Add tests (where practical) for:

   • Prompt compaction
   • Token estimation
   • Cache key stability
   • Chunking logic

Deliverables:

• New utility modules (e.g., services/llmOrchestrator.ts, services/tokenEstimator.ts, services/llmCache.ts)
• Updated types to store token metrics and chapter summaries.
• Updated storageService to persist new data.
• Updated components (minimal) to surface metrics.

Acceptance criteria:

• Repeated identical actions result in cache hits and no additional API calls.
• Regenerating a chapter with guidance transmits significantly less text than before (measured by estimator).
• Market report / outline generation produce same quality but with shorter prompts.
• No breaking changes to existing app flows.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: analyze the codes line by line and create a detailed blueprint and todo

Assistant: Below is a codebase analysis + blueprint + TODO plan for crazyrob425/KDP-E-Book-Generator based on the files I could pull from the repo (root, services/, server/, electron/, hooks/, plus directory listings under components/). Where I say “line-by-line”, I’m going file-by-file and section-by-section (the repo has a very large App.tsx, so I’m describing it in contiguous “blocks” of lines / responsibilities rather than literally enumerating 45k lines one by one).

---

0) High-level: what this repo currently is

Current architecture (as implemented):

• React + Vite + TypeScript frontend (mostly driven by a single massive App.tsx orchestrator).
• AI generation via Google Gemini (client-side calls through @google/genai) in services/geminiService.ts.
• Local persistence:
  • “V5 DB” using IndexedDB wrapper in services/storageService.ts.
  • Extra “AutoSave” feature saving a JSON file via Electron IPC if running as desktop (hooks/useAutoSave.ts).
• Electron wrapper:
  • electron/main.ts sets up a frameless window + IPC handlers.
  • Bridges to “bot automation” and “market research workers”.
• Optional Node server:
  • server/server.ts provides WebSocket-based automation sessions.
  • server/automation-worker.ts Playwright automation for KDP upload flow (with CAPTCHA yield/continue).
  • server/market-research-worker.ts fetches Google Trends + scrapes Amazon results + Amazon suggest.

Big gaps (blocking “Binary Books E‑Crafter” spec):

• EPUB generation is currently stubbed/disabled (services/epubGenerator.ts returns a plain text Blob).
• The “multi-agent swarm”, lore engine, continuity, long-form pipeline, and robust formatting checks are not implemented yet (only single-pass prompts exist).
• Scraping/automation will be fragile and has compliance/risk concerns (Amazon ToS, bot detection, CAPTCHA handling, account lock risk).

---

1) Repository map (what’s here)

Root

• App.tsx (very large): main UI, state machine, orchestration of steps, saving, generation triggers.
• index.tsx, `index.htm...

This pull request was created from Copilot chat.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.