⚡ Copilot Token Optimization2026-08-23 — duplicate-code-detector

[github-actions]

Target Workflow: duplicate-code-detector

Source report: #7653
Estimated cost per run: $0.00 (reported), actual model gpt-5.4-mini
Total tokens per run: ~162K (single run, run_id 32600239132)
Cache/working-set signal: rebuild_factor 4.49 (cumulative input 161,106 / peak input 35,863) — the agent is re-sending ~4.5x more input tokens across turns than its single largest turn needs, indicating poor turn-to-turn context reuse
LLM turns (invocations): 5

Current Configuration

Setting	Value
Tools loaded	github (toolset: issues), bash: true (unrestricted shell)
Tools actually used	Not captured in run data (tool_usage/mcp_tool_usage absent from log JSON for this run) — prompt instructs the agent to rely on pre-computed files via cat/sed, not GitHub MCP tools, for the existing-issues phase
Network groups	github only — already minimal
Pre-agent steps	Yes — jscpd install/run, file metrics, grep pattern analysis, existing-issue lookup via gh issue list (good practice already)
Prompt size	7,405 chars (.github/workflows/duplicate-code-detector.md)

This workflow already follows several best practices (pre-agent steps:, capped create-issue.max: 3, minimal network.allowed, restricted github.toolsets: [issues]), so the main lever is turn/context efficiency, not tool surface.

Recommendations

1. Cap and structure repeated file reads to cut turn count

Estimated savings: ~40-50K tokens/run (~25-30%)

rebuild_factor 4.49 over only 5 invocations means each turn is re-including large chunks of previously-seen context (jscpd JSON, code metrics, grep output) rather than referencing them once. The prompt already tells the agent to cat these files, but doesn't instruct it to read each exactly once and hold results in-turn. Add an explicit constraint:

## Efficiency Constraint

Read each pre-computed file (`code-metrics.txt`, `jscpd-top.json`, `grep-analysis.txt`,
`existing-issues.json`) **exactly once**, at the start of your analysis. Do not re-cat
or re-fetch these files in later turns — retain the content from your first read.
Target completing the full analysis (read → prioritize → file issues) in ≤3 turns.

Also tighten the existing "Complete your analysis in ≤4 turns" line (line 126) down to ≤3, since 5 turns were used and the task is fully pre-computed.

2. Trim jscpd-top.json further and drop unused code-metrics.txt fields if not consumed

Estimated savings: ~10-15K tokens/run (~7-9%)

The pre-agent step already caps jscpd output to top 15 duplicates (good), but firstFile/secondFile objects include name, start, end for both sides per finding — for 15 findings this is still sizable JSON. Since the prompt only needs file+line-range evidence, consider flattening to a terser array in the jq transform:

jq '{
  statistics: {total: .statistics.total, percentage: .statistics.percentage},
  duplicates: (.duplicates | sort_by(-.lines) | .[0:10]
    | map({lines, tokens,
           a: "\(.firstFile.name):\(.firstFile.start)-\(.firstFile.end)",
           b: "\(.secondFile.name):\(.secondFile.start)-\(.secondFile.end)"}))
}' /tmp/gh-aw/jscpd-src/jscpd-report.json > /tmp/gh-aw/jscpd-top.json

Reducing top-N from 15 → 10 (the workflow already only reports findings scoring ≥4, so extra low-value findings are rarely used) and flattening file refs to single strings both reduce JSON verbosity.

3. Move "Check existing issues" filtering logic fully into the pre-agent step

Estimated savings: ~5-8K tokens/run (~4%)

Currently the agent is handed the full existing-issues.json and must reason over open/closed/stateReason per candidate title (Phase 5, lines 128-136). This filtering is deterministic and could be done in the pre-agent step by pre-computing a simple denylist of blocked titles, so the agent only needs a short list instead of the full issue JSON with state metadata:

# In "Check existing duplicate issues" step, after fetching:
jq -r '.[] | select(.state=="OPEN" or .stateReason=="not_planned") | .title' \
  /tmp/gh-aw/existing-issues.json > /tmp/gh-aw/blocked-titles.txt

Then instruct the agent: "Skip any finding whose title matches a line in /tmp/gh-aw/blocked-titles.txt" — a much smaller artifact than the full JSON.

4. Confirm model choice remains appropriate

No change recommended here, but note for tracking: gpt-5.4-mini is already the lower-cost model tier for this workflow, so further savings must come from token volume, not model swap.

Expected Impact

Metric	Current	Projected	Savings
Total tokens/run	~162K	~105-115K	~30-35%
LLM turns	5	≤3	-2
Rebuild factor	4.49	~2.5-3.0 (est.)	~35-45%

Implementation Checklist

• Add "Efficiency Constraint" section instructing single-pass file reads (recommendation 1)
• Lower turn cap from ≤4 to ≤3 in "Scope Constraint" section
• Reduce jscpd top-N from 15 to 10 and flatten file location fields in the jq transform (recommendation 2)
• Pre-compute blocked-titles.txt denylist in the "Check existing duplicate issues" step and update Phase 5 instructions to reference it instead of the full JSON (recommendation 3)
• Recompile: gh aw compile .github/workflows/duplicate-code-detector.md
• Post-process: npx tsx scripts/ci/postprocess-smoke-workflows.ts
• Verify CI passes on next scheduled/dispatch run
• Compare token usage on new run vs. baseline (~162K tokens, 5 turns)

Generated by Daily Copilot Token Optimization Advisor · auto · 41.9 AIC · ⊞ 10.7K · ◷

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #7689

View newer issue

---

This action was performed automatically by the Daily Copilot Token Optimization Advisor workflow.