Skip to content

docs(sdk-expert): require a cited source before answering behaviour questions - #26

Merged
kherembourg merged 4 commits into
mainfrom
docs/sdk-expert-source-of-truth
Sep 2, 2026
Merged

kherembourg merged 4 commits into
mainfrom
docs/sdk-expert-source-of-truth

Conversation

@kherembourg

Copy link
Copy Markdown
Contributor

The incident

A client reported "campaign capping does not work". The rule is that capping (impression cap, frequency, exposure window) applies only to campaigns delivered by an event trigger such as APP_STARTED. A campaign served through a Placement is never capped, by design.

That answer was already available in three places:

  • publicly, in campaign-configuration.md on docs.purchasely.com,
  • twice in references/concepts/campaigns.md (the > Important callout under The four campaign dimensions, and the capping bullet under Anti-patterns; lines 38 and 149 before this change),
  • in the triage note, posed as a question on the first message of the conversation.

The investigation read the triage note, then spent a day rediscovering the rule by reading the Android SDK, the iOS SDK, the Rails backend and the Cloudflare worker, and concluded "product bug" on documented behaviour.

purchasely-sdk-expert is the free-form Q&A skill, and it was the only one of the five with no documentation-verification rule (0 occurrences of docs.purchasely in both its SKILL.md and its agent file, while purchasely-debug, purchasely-integrate, purchasely-migrate and purchasely-review all carry one).

What changes

purchasely/skills/purchasely-sdk-expert/SKILL.md

  • Source of truth section, before ## Core context: search references/ first and cite path:line as read at answer time; go to https://docs.purchasely.com/ when the references do not answer, look dated, or when the answer depends on an exact signature or on current Console behaviour; say "I do not know" and name where to look rather than produce a plausible answer; do not read SDK, backend or Console source code to discover expected behaviour; qualify before accusing, and quote the source in the ticket.
  • Routing index table at the top: topic to reference file, one row per file in references/concepts/ plus the cross-store, versions, Console and architecture references.
  • Answering workflow step 1 now looks up the documented behaviour before it classifies. A report that something does not work was previously routed straight to purchasely-debug, which reproduced the incident inside the skill; the handover now happens only when the observed behaviour diverges from the documented one.
  • The Campaigns section states the trigger-only capping rule with its source.
  • The response format requires a source for every statement about expected behaviour.

purchasely/agents/purchasely-sdk-expert.md carries the same Source of truth rule, and its fallback paragraph now forbids answering a product behaviour rule without a source.

purchasely/references/concepts/campaigns.md repeats the trigger-only capping rule in the opening summary, so it is visible without reading the whole file. No new content: the rule was already correct further down.

Verification

Dry run of "campaign capping does not work on Android" against the modified skill, three steps, no source code:

  1. Top of SKILL.md: a symptom report is not automatically debug work, look the topic up in the routing index.
  2. Routing index, first row: Campaign, capping, frequency cap, impression cap, exposure window, APP_STARTED trigger, campaign not displayed to concepts/campaigns.md.
  3. campaigns.md opening summary: capping applies to trigger-based delivery only, a placement-served campaign is never capped, this is expected behaviour.

The skill's own Campaigns section gives the same answer with its source in one step for a reader that gets there first.

No name: or description: frontmatter changed, so no agentskill.sh re-scan is needed. No version bump; the CHANGELOG entry sits under [Unreleased].

🤖 Generated with Claude Code

kherembourg and others added 2 commits September 2, 2026 08:50
…uestions

A support investigation spent a day reading the Android SDK, the iOS SDK, the
Rails backend and the Cloudflare worker to rediscover a documented product
rule: campaign capping applies to trigger-based delivery only, never to a
campaign served through a Placement. The answer was already in
references/concepts/campaigns.md, in the public docs, and in the triage note.

- add a Source of truth rule to the skill and the agent: references first with
  a cited path:line, then docs.purchasely.com, then "I do not know" plus where
  to look; never answer a behaviour rule from memory; source code explains a
  gap from documented behaviour, it never defines it
- add a routing index table (topic to reference file) at the top of the skill
- look up documented behaviour before classifying the question, so a symptom
  report is not handed to purchasely-debug before the rule is checked
- state the trigger-only capping rule in the skill Campaigns section and in the
  campaigns.md opening summary

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…divergence

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Sep 2, 2026 •

Copy link
Copy Markdown

Greptile Summary

The PR strengthens the SDK expert’s documentation-first behavior and expands its topic routing.

  • Requires cited references or official documentation for expected-behavior answers.
  • Routes behavior questions to focused references before debugging or source inspection.
  • Documents trigger-only campaign capping prominently.
  • Adds a dedicated end-to-end architecture route alongside the client architecture-pattern route.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported architecture-routing omission is fixed by the dedicated end-to-end architecture row and its valid bundled reference.

Important Files Changed

Filename Overview
purchasely/skills/purchasely-sdk-expert/SKILL.md Adds documentation-first answer rules, comprehensive reference routing, corrected architecture routing, and campaign-capping guidance.
purchasely/agents/purchasely-sdk-expert.md Aligns the Claude agent wrapper with the skill’s source-verification and behavior-question requirements.
purchasely/hooks/intro.md Expands session-start routing so behavior reports reach the expert and concept references before source inspection.
purchasely/references/concepts/campaigns.md Promotes the existing trigger-only campaign-capping rule into the opening summary.
CHANGELOG.md Records the documentation-first behavior, routing expansion, and campaign guidance under Unreleased.

Reviews (2): Last reviewed commit: "fix(sdk-expert): address review comments..." | Re-trigger Greptile

Comment thread purchasely/skills/purchasely-sdk-expert/SKILL.md
The description only described SDK questions, so a product or Console
behaviour question ("how does campaign capping work", "the capping does not
work") did not reliably invoke the skill. A subagent that investigates a
report sees the description only, never the session-start hook text, so the
SDK framing was the only signal it had.

- description of the skill and the agent now covers product and Console
  behaviour (campaign triggers and capping, audiences, placements and screen
  resolution, A/B tests, running modes, cache, offer eligibility,
  localization), a question asked by another agent, and "how does X work" /
  "why does X happen" / "X does not work" reports
- hooks/intro.md lists the same concept keywords, and routes a report of
  something not working to the skill and to references/concepts/ before any
  source code

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kherembourg

Copy link
Copy Markdown
Contributor Author

Follow-up: the trigger itself was too narrow

Widening the rules was not enough, because the skill did not reliably get invoked in the first place. Two trigger surfaces exist, and both were scoped to SDK questions:

  1. the description: frontmatter of skills/purchasely-sdk-expert/SKILL.md and agents/purchasely-sdk-expert.md,
  2. the session-start hook text in hooks/intro.md.

Three gaps:

Gap Cause
A product or Console behaviour question (capping, audience not matched, placement priority, A/B) Both texts said SDK. "How does campaign capping work" does not read as an SDK question, so the skill was not loaded. This is the incident.
A question with no Purchasely keyword "The capping does not work" matched none of paywalls, subscriptions, PLYPresentation, userLogin.
An agent investigating a report inside a subagent The SessionStart hook fires on startup|clear|compact in the main session only. A subagent sees the description: and nothing else.

Both descriptions now cover product and Console behaviour (campaign triggers and capping, audiences and targeting, placements and screen resolution, A/B tests, running modes, presentation cache, offer eligibility, localization), a question coming from another agent, and the how does X work / why does X happen / X does not work phrasings, with the instruction to check the skill before reading SDK, backend or Console source code. hooks/intro.md carries the same keyword list and routes a "does not work" report to the skill and to references/concepts/ first.

⚠️ This changes a description: field, so the agentskill.sh re-scan described in CLAUDE.md must run once this is merged on main.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A few new/updated instructions and examples are internally inconsistent with the PR’s own “cite path:line / behaviour spelling consistency” requirements and should be aligned before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR hardens the purchasely-sdk-expert documentation/Q&A skill to require verifying and citing a documented source before stating expected Purchasely product/SDK behaviour, aiming to prevent “rediscovering” documented rules via source-code spelunking (notably campaign capping vs placements).

Changes:

  • Added a “Source of truth” policy and a routing index to purchasely-sdk-expert to force reference/docs lookups and citations before answering behaviour questions.
  • Documented the trigger-only campaign capping rule more prominently (including a top-of-file summary in the campaigns reference).
  • Updated the agent wrapper, hooks intro routing, and CHANGELOG.md to reflect the new behaviour-first verification flow.
File summaries
File Description
purchasely/skills/purchasely-sdk-expert/SKILL.md Adds source-of-truth rules, routing index, and strengthens behaviour-verification/citation requirements (including campaign capping guidance).
purchasely/references/concepts/campaigns.md Elevates the trigger-only capping rule into the opening summary for faster discovery.
purchasely/hooks/intro.md Updates session-start routing guidance so “does not work” reports route to references/skill before debugging code.
purchasely/agents/purchasely-sdk-expert.md Mirrors the source-of-truth rule in the Claude Code agent wrapper and tightens fallback behaviour.
CHANGELOG.md Records the user-visible documentation/skill behaviour changes under [Unreleased].
Review details

Suppressed comments (1)

purchasely/skills/purchasely-sdk-expert/SKILL.md:204

  • The new capping rule is great to have inline, but the citation currently points only to the file/sections. Since this skill now requires path:line citations for expected behaviour, it would help to reference the specific lines in references/concepts/campaigns.md that state the rule (so agents can cite them verbatim).
- **Capping (impression cap, frequency, exposure window) applies to trigger-based delivery only.** A campaign served through a Placement is never capped: the SDK evaluates it every time the app displays that placement. "The capping does not work" on a placement-served campaign is documented behaviour, not a defect. Source: `../../references/concepts/campaigns.md` (the `> Important` callout under "The four campaign dimensions", and the capping bullet under "Anti-patterns").
  • Files reviewed: 5/5 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread purchasely/skills/purchasely-sdk-expert/SKILL.md Outdated
Comment thread purchasely/skills/purchasely-sdk-expert/SKILL.md
…ilure

- routing index: add purchasely-architecture.md, the end-to-end SDK, server,
  store and backend reference that only architecture-patterns.md covered
  (Greptile P2)
- Source of truth: define what counts as a source, a references path:line or
  a docs.purchasely.com page, and exclude the skill file itself, source files
  and earlier answers (Copilot)
- spelling: use "behavior" in the skill, the agent and the hook intro, the
  spelling those files already used (Copilot)
- references/concepts/screen-resolution.md: emphasis on line 13 switched to
  asterisks, which sets one MD049 style for the file and clears the 16
  pre-existing markdownlint errors that failed CI

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kherembourg

Copy link
Copy Markdown
Contributor Author

All review findings addressed in 4ff7293, plus the red CI.

# Source Finding Outcome
1 Greptile P2 Routing index omits purchasely-architecture.md Fixed. Added as its own row above architecture-patterns.md, worded by layer (SDK, Purchasely Server, stores, client backend, third-party tools). All five top-level reference files are now routed.
2 Copilot "Source of truth" does not say what counts as a source Fixed. A source is a file under ../../references/ cited as path:line, or a page on https://docs.purchasely.com/, and nothing else. The skill file itself, an SDK or backend source file, and an earlier answer are explicitly excluded.
3 Copilot "behavior" / "behaviour" mixed in the response-format checklist Fixed. The file used US "behavior" before this PR and my additions brought 16 UK spellings; normalized to "behavior" in the skill, the agent and hooks/intro.md. references/concepts/campaigns.md keeps "behaviour", the spelling it already used. A repo-wide rename (50 UK vs 26 US) stays out of scope.
4 CI, Lint Markdown 16 MD049 errors, pre-existing on main Fixed. references/concepts/screen-resolution.md:13 used **_Everyone else_**, which set the file MD049 style to underscore and made the 8 asterisk emphases below it fail (counted twice, references/ is a symlink to purchasely/references/). Switched that one emphasis to asterisks. Verified locally with the CI version: markdownlint-cli2@0.14.0, 138 files, 0 errors.

@kherembourg
kherembourg merged commit 81ab276 into main Sep 2, 2026
3 checks passed
@kherembourg
kherembourg deleted the docs/sdk-expert-source-of-truth branch September 2, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants