docs(sdk-expert): require a cited source before answering behaviour questions - #26
Conversation
…uestions A support investigation spent a day reading the Android SDK, the iOS SDK, the Rails backend and the Cloudflare worker to rediscover a documented product rule: campaign capping applies to trigger-based delivery only, never to a campaign served through a Placement. The answer was already in references/concepts/campaigns.md, in the public docs, and in the triage note. - add a Source of truth rule to the skill and the agent: references first with a cited path:line, then docs.purchasely.com, then "I do not know" plus where to look; never answer a behaviour rule from memory; source code explains a gap from documented behaviour, it never defines it - add a routing index table (topic to reference file) at the top of the skill - look up documented behaviour before classifying the question, so a symptom report is not handed to purchasely-debug before the rule is checked - state the trigger-only capping rule in the skill Campaigns section and in the campaigns.md opening summary Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…divergence Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
| Filename | Overview |
|---|---|
| purchasely/skills/purchasely-sdk-expert/SKILL.md | Adds documentation-first answer rules, comprehensive reference routing, corrected architecture routing, and campaign-capping guidance. |
| purchasely/agents/purchasely-sdk-expert.md | Aligns the Claude agent wrapper with the skill’s source-verification and behavior-question requirements. |
| purchasely/hooks/intro.md | Expands session-start routing so behavior reports reach the expert and concept references before source inspection. |
| purchasely/references/concepts/campaigns.md | Promotes the existing trigger-only campaign-capping rule into the opening summary. |
| CHANGELOG.md | Records the documentation-first behavior, routing expansion, and campaign guidance under Unreleased. |
Reviews (2): Last reviewed commit: "fix(sdk-expert): address review comments..." | Re-trigger Greptile
The description only described SDK questions, so a product or Console
behaviour question ("how does campaign capping work", "the capping does not
work") did not reliably invoke the skill. A subagent that investigates a
report sees the description only, never the session-start hook text, so the
SDK framing was the only signal it had.
- description of the skill and the agent now covers product and Console
behaviour (campaign triggers and capping, audiences, placements and screen
resolution, A/B tests, running modes, cache, offer eligibility,
localization), a question asked by another agent, and "how does X work" /
"why does X happen" / "X does not work" reports
- hooks/intro.md lists the same concept keywords, and routes a report of
something not working to the skill and to references/concepts/ before any
source code
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up: the trigger itself was too narrowWidening the rules was not enough, because the skill did not reliably get invoked in the first place. Two trigger surfaces exist, and both were scoped to SDK questions:
Three gaps:
Both descriptions now cover product and Console behaviour (campaign triggers and capping, audiences and targeting, placements and screen resolution, A/B tests, running modes, presentation cache, offer eligibility, localization), a question coming from another agent, and the
|
There was a problem hiding this comment.
🟡 Changes recommended
A few new/updated instructions and examples are internally inconsistent with the PR’s own “cite path:line / behaviour spelling consistency” requirements and should be aligned before merging.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR hardens the purchasely-sdk-expert documentation/Q&A skill to require verifying and citing a documented source before stating expected Purchasely product/SDK behaviour, aiming to prevent “rediscovering” documented rules via source-code spelunking (notably campaign capping vs placements).
Changes:
- Added a “Source of truth” policy and a routing index to
purchasely-sdk-expertto force reference/docs lookups and citations before answering behaviour questions. - Documented the trigger-only campaign capping rule more prominently (including a top-of-file summary in the campaigns reference).
- Updated the agent wrapper, hooks intro routing, and
CHANGELOG.mdto reflect the new behaviour-first verification flow.
File summaries
| File | Description |
|---|---|
| purchasely/skills/purchasely-sdk-expert/SKILL.md | Adds source-of-truth rules, routing index, and strengthens behaviour-verification/citation requirements (including campaign capping guidance). |
| purchasely/references/concepts/campaigns.md | Elevates the trigger-only capping rule into the opening summary for faster discovery. |
| purchasely/hooks/intro.md | Updates session-start routing guidance so “does not work” reports route to references/skill before debugging code. |
| purchasely/agents/purchasely-sdk-expert.md | Mirrors the source-of-truth rule in the Claude Code agent wrapper and tightens fallback behaviour. |
| CHANGELOG.md | Records the user-visible documentation/skill behaviour changes under [Unreleased]. |
Review details
Suppressed comments (1)
purchasely/skills/purchasely-sdk-expert/SKILL.md:204
- The new capping rule is great to have inline, but the citation currently points only to the file/sections. Since this skill now requires
path:linecitations for expected behaviour, it would help to reference the specific lines inreferences/concepts/campaigns.mdthat state the rule (so agents can cite them verbatim).
- **Capping (impression cap, frequency, exposure window) applies to trigger-based delivery only.** A campaign served through a Placement is never capped: the SDK evaluates it every time the app displays that placement. "The capping does not work" on a placement-served campaign is documented behaviour, not a defect. Source: `../../references/concepts/campaigns.md` (the `> Important` callout under "The four campaign dimensions", and the capping bullet under "Anti-patterns").
- Files reviewed: 5/5 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ilure - routing index: add purchasely-architecture.md, the end-to-end SDK, server, store and backend reference that only architecture-patterns.md covered (Greptile P2) - Source of truth: define what counts as a source, a references path:line or a docs.purchasely.com page, and exclude the skill file itself, source files and earlier answers (Copilot) - spelling: use "behavior" in the skill, the agent and the hook intro, the spelling those files already used (Copilot) - references/concepts/screen-resolution.md: emphasis on line 13 switched to asterisks, which sets one MD049 style for the file and clears the 16 pre-existing markdownlint errors that failed CI Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
All review findings addressed in
|
The incident
A client reported "campaign capping does not work". The rule is that capping (impression cap, frequency, exposure window) applies only to campaigns delivered by an event trigger such as
APP_STARTED. A campaign served through a Placement is never capped, by design.That answer was already available in three places:
campaign-configuration.mdon docs.purchasely.com,references/concepts/campaigns.md(the> Importantcallout under The four campaign dimensions, and the capping bullet under Anti-patterns; lines 38 and 149 before this change),The investigation read the triage note, then spent a day rediscovering the rule by reading the Android SDK, the iOS SDK, the Rails backend and the Cloudflare worker, and concluded "product bug" on documented behaviour.
purchasely-sdk-expertis the free-form Q&A skill, and it was the only one of the five with no documentation-verification rule (0 occurrences ofdocs.purchaselyin both itsSKILL.mdand its agent file, whilepurchasely-debug,purchasely-integrate,purchasely-migrateandpurchasely-reviewall carry one).What changes
purchasely/skills/purchasely-sdk-expert/SKILL.md## Core context: searchreferences/first and citepath:lineas read at answer time; go to https://docs.purchasely.com/ when the references do not answer, look dated, or when the answer depends on an exact signature or on current Console behaviour; say "I do not know" and name where to look rather than produce a plausible answer; do not read SDK, backend or Console source code to discover expected behaviour; qualify before accusing, and quote the source in the ticket.references/concepts/plus the cross-store, versions, Console and architecture references.purchasely-debug, which reproduced the incident inside the skill; the handover now happens only when the observed behaviour diverges from the documented one.purchasely/agents/purchasely-sdk-expert.mdcarries the same Source of truth rule, and its fallback paragraph now forbids answering a product behaviour rule without a source.purchasely/references/concepts/campaigns.mdrepeats the trigger-only capping rule in the opening summary, so it is visible without reading the whole file. No new content: the rule was already correct further down.Verification
Dry run of "campaign capping does not work on Android" against the modified skill, three steps, no source code:
SKILL.md: a symptom report is not automatically debug work, look the topic up in the routing index.Campaign, capping, frequency cap, impression cap, exposure window, APP_STARTED trigger, campaign not displayedtoconcepts/campaigns.md.campaigns.mdopening summary: capping applies to trigger-based delivery only, a placement-served campaign is never capped, this is expected behaviour.The skill's own Campaigns section gives the same answer with its source in one step for a reader that gets there first.
No
name:ordescription:frontmatter changed, so no agentskill.sh re-scan is needed. No version bump; the CHANGELOG entry sits under[Unreleased].🤖 Generated with Claude Code