chore: sync skills (agent-skills-v0.922.0, context-mill@v1.51.0)

.claude-plugin/marketplace.json

@@ -12,7 +12,7 @@
       "displayName": "PostHog",
       "source": "./",
       "description": "Access PostHog analytics, feature flags, experiments, error tracking, and insights directly from your AI coding tool. Optionally capture Claude Code sessions to PostHog LLM Analytics.",
-      "version": "1.1.62",
+      "version": "1.1.63",
       "author": {
         "name": "PostHog",
         "email": "hey@posthog.com",

.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "posthog",
   "description": "Access PostHog analytics, feature flags, experiments, error tracking, and insights directly from your AI coding tool. Optionally capture Claude Code sessions to PostHog LLM Analytics.",
-  "version": "1.1.62",
+  "version": "1.1.63",
   "author": {
     "name": "PostHog",
     "email": "hey@posthog.com",

.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "posthog",
-  "version": "1.0.60",
+  "version": "1.0.61",
   "description": "Access PostHog analytics, feature flags, experiments, error tracking, and insights directly from Codex",
   "author": {
     "name": "PostHog",

.cursor-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "posthog",
   "displayName": "PostHog",
-  "version": "1.1.56",
+  "version": "1.1.57",
   "description": "Access PostHog analytics, feature flags, experiments, error tracking, and insights directly from Cursor",
   "author": {
     "name": "PostHog",

gemini-extension.json

@@ -1,6 +1,6 @@
 {
   "name": "posthog",
-  "version": "1.0.58",
+  "version": "1.0.59",
   "description": "Access PostHog analytics, feature flags, experiments, error tracking, and insights directly from Gemini CLI",
   "mcpServers": {
     "posthog": {

skills/authoring-scouts/references/dedupe-and-memory.md

@@ -10,7 +10,7 @@ Every scout classifies each candidate finding against prior runs, the inbox, and
 Bake this classifier into the scout's Decide section:
 
 1. **Net new** — no prior run mentions the topic, no inbox report and no scratchpad entry covers it. → Author a report via `emit_report` if it clears the report bar (see [`report-contract.md`](report-contract.md)).
-2. **Material update on an existing live report** — a live report already covers the topic (one this scout authored last run, or a pipeline report), but there's new evidence (a different corroborating source, a fresh deploy correlation, contradicting data, a meaningful escalation in scope). → **`edit_report` it** — `append_note` with the fresh evidence, or rewrite `title`/`summary` on a report the scout authored.
+2. **Material update on an existing live report** — a live report already covers the topic (one this scout authored last run, or a pipeline report), but there's new evidence (a different corroborating source, a fresh deploy correlation, contradicting data, a meaningful escalation in scope). → **`edit_report` it** — use `append_evidence` for the new observation, `append_note` for a reading of it, or rewrite `title`/`summary` on a report the scout authored.
    Don't mint a near-duplicate.
    **Live reports only:** `edit_report` never changes a report's status, so if the prior report is suppressed or resolved and the issue is genuinely back, author a **fresh** report (citing the prior `report_id` in the summary) rather than editing a closed one nobody will see.
 3. **Same fact already covered** — an existing report already captures the same evidence shape, nothing has changed. → Skip.

skills/authoring-scouts/references/report-contract.md

@@ -231,15 +231,21 @@ The fleet's reviewer map should compound over time.
 
 ## `edit_report` — update an existing report
 
-Rewrite `title`/`summary`, append a note, set `suggested_reviewers`, and/or replace `charts` / `suggested_prompts` on a report that already exists.
-Pass `run_id` (the current run) and `report_id`, plus at least one of `title`, `summary`, `append_note`, `suggested_reviewers`, `charts`, `suggested_prompts`.
-An edit that supplies content (`title`, `summary`, `charts`, `suggested_prompts`, `append_note`, or a reviewer `reason`) passes the same safety judge as `emit_report`; an unsafe edit is rejected whole and the report keeps what it had.
+Rewrite `title`/`summary`, append evidence or a note, set `suggested_reviewers`, and/or replace `charts` / `suggested_prompts` on a report that already exists.
+Pass `run_id` (the current run) and `report_id`, plus at least one of `title`, `summary`, `append_note`, `append_evidence`, `suggested_reviewers`, `charts`, `suggested_prompts`.
+An edit that supplies content (`title`, `summary`, `charts`, `suggested_prompts`, `append_note`, `append_evidence`, or a reviewer `reason`) passes the same safety judge as `emit_report`; an unsafe edit is rejected whole and the report keeps what it had.
 
 `edit_report` can target **any** of the team's inbox reports — not just ones a scout authored.
 That makes it the right tool when a later run learns something about a report the pipeline (or another scout) created.
 Rules of good behavior:
 
-- **Prefer `append_note` over rewriting** `title`/`summary` on a report you didn't author.
+- Use **`append_evidence`** for a new observation that a reader can check.
+  It takes the same `{description, source_id}` items as `emit_report`, and each one lands in the report's evidence rail as a bound signal, so the report's `signal_count` and `total_weight` grow with it.
+- Use **`append_note`** for commentary — a reading of the report that adds nothing to check, such as the owning team already knowing, or a deploy having fixed it.
+  Send both in one call when an observation needs a reading alongside it.
+- **A recovery is a note, not evidence.** `signal_count` and `total_weight` only grow, and both feed the inbox ranking, so evidence that an issue is over would rank the report as stronger.
+- **At the cap, the note is the channel that still lands.** Emit plus every append share the report's **50** evidence rows, and the grouping pipeline can raise the count too, so a long-lived report can fill up. An append past the cap is rejected and the report keeps what it had.
+- Prefer these additive fields over rewriting `title`/`summary` on a report you didn't author.
   A note is additive and audit-friendly (it carries your scout as the author); a rewrite silently overwrites a human- or pipeline-authored headline.
 - **Don't fight an in-flight pipeline.** A report the summary/research workflow is mid-run on can have its fields overwritten under you.
   If a report is actively being worked, append a note rather than rewriting.

skills/authoring-scouts/references/scout-patterns.md

@@ -390,8 +390,8 @@ So the trigger for this pattern is any of: **a judgment with more than one axis*
 - **Bound what you write for non-candidates.** "Record which axis failed" is right for items that are close, and ruinous as a blanket rule on a busy queue — one `remember` call per rejected item can spend the run before the real candidates get read.
   Persist a **state transition** (an item that changed axis since last run) or a capped set of near-misses, and roll the rest into one aggregate backlog entry.
 - **Close the loop on what you filed — and know what closing it can and cannot do.** A "ready to pick up" report is wrong the moment someone picks it up, and it costs a person duplicating work already underway.
-  Re-check each `report:` entry every run and `edit_report` once the item is assigned, PR-linked, or closed — but note that `edit_report` mutates `title`, `summary`, `append_note`, `suggested_reviewers`, `charts`, and `suggested_prompts` **only**.
-  It cannot change status or actionability, so an appended note does not retire the report.
+  Re-check each `report:` entry every run and `edit_report` once the item is assigned, PR-linked, or closed — but note that `edit_report` mutates `title`, `summary`, `append_note`, `append_evidence`, `suggested_reviewers`, `charts`, and `suggested_prompts` **only**.
+  It cannot change status or actionability, so an appended note or evidence row does not retire the report.
   Rewrite the **title and summary** so the stale framing is gone from the surface a human scans, and leave the status change to a person.
 - **Routing the outcome is part of the design.** On the report channel a queue scout can hand work straight to a draft PR: `actionability: immediately_actionable` + `repository` + a `priority` makes the report **eligible** to autostart one.
   Eligible is not automatic — the team's autostart toggle, its priority threshold, the org's self-driving quota, and resolving a runner identity each gate it independently, so a correctly-filed report can sit still for reasons that have nothing to do with the scout.

skills/creating-online-evaluations/SKILL.md

@@ -2,15 +2,15 @@
 name: creating-online-evaluations
 description: >
   Author continuously-running online evaluations in PostHog AI observability, grounded in real failure
-  modes you've identified. Use when the user wants evaluations that automatically score new generations
-  or whole traces going forward — "create an eval to catch X", "continuously check that responses do Y",
+  modes you've identified. Use when the user wants evaluations that automatically score new generations or
+  whole traces going forward — "create an eval to catch X", "continuously check that responses do Y",
   "turn these failures into evals". Covers letting the explored data decide how many evals to create,
-  proposing that set in plain language and asking the user which ones they want, choosing the target and
-  eval type (hog / llm_judge / sentiment), configuring a provider, model, and usable provider key for an
-  llm_judge eval, scoping which generations trigger it via conditions, creating disabled, verifying scope,
-  and enabling. Falls back to proposing a sentiment eval when no failure mode is worth catching.
-  Finding and ranking the failure modes worth evaluating is its own job — use exploring-ai-failures first.
-  To debug or manage evaluations that already exist, use exploring-llm-evaluations.
+  proposing that set for the user to pick, choosing the target and eval type (hog / llm_judge /
+  sentiment), configuring a provider and model for an llm_judge eval (a provider key gates enabling, not
+  creation), scoping which generations trigger it via conditions, creating disabled, verifying scope, and
+  enabling. Proposes a sentiment eval when no failure mode is worth catching. Finding and ranking the
+  failure modes worth evaluating is its own job — use exploring-ai-failures first. To debug or manage
+  evaluations that already exist, use exploring-llm-evaluations.
 ---
 
 # Creating online evaluations
@@ -129,11 +129,11 @@ set into production, which is noise and (for a judge) cost the user didn't agree
 
 ### 2.1 — Choose the eval type
 
-| Use…        | When the criterion is…                                                                                                                |
-| ----------- | ------------------------------------------------------------------------------------------------------------------------------------- |
-| `hog`       | Structural / rule-based (JSON parses, length, regex, tool-call shape). Cheap, deterministic, **no provider key needed.**              |
-| `llm_judge` | Subjective / fuzzy (tone, factuality, on-topic). Costs an LLM call per run; needs a provider, model, and usable provider key.         |
-| `sentiment` | You want sentiment labels on user messages, not a pass/fail (unless very specifically asked for, usually not relevant to this skill). |
+| Use…        | When the criterion is…                                                                                                                                                            |
+| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `hog`       | Structural / rule-based (JSON parses, length, regex, tool-call shape). Cheap, deterministic, **no provider key needed.**                                                          |
+| `llm_judge` | Subjective / fuzzy (tone, factuality, on-topic). Costs an LLM call per run; needs a provider and model. A usable provider key is only needed to enable it, not to create a draft. |
+| `sentiment` | You want sentiment labels on user messages, not a pass/fail (unless very specifically asked for, usually not relevant to this skill).                                             |
 
 Reach for `hog` first, escalate to `llm_judge` if there is no deterministic way to check for what we want to check.
 
@@ -210,12 +210,14 @@ when exact structure matters.
 
 ### 2.3 — Configure the LLM judge
 
-An `llm_judge` evaluation requires a valid `provider` and `model`. It also needs a usable provider key
-when it runs. `provider_key_id` controls whether the evaluation pins one specific key:
+An `llm_judge` evaluation requires only a valid `provider` and `model` to create as a draft. A usable
+provider key is needed to enable it, not to save it, so no `ok` key is a reason to keep the eval disabled,
+not a reason to stop. `provider_key_id` controls whether the evaluation pins one specific key:
 
 - Set `provider_key_id` to the UUID of an `ok`-state key for the same provider to pin it.
-- Set `provider_key_id` to `null` to use the team's active provider key. The active key must be in the
-  `ok` state and use the same provider as `model_configuration.provider`.
+- Set `provider_key_id` to `null` to use the team's active provider key. This is always a valid saved
+  value. To enable the eval, the active key must be in the `ok` state and use the same provider as
+  `model_configuration.provider`.
 
 Hog and sentiment evaluations skip this step.
 
@@ -229,10 +231,12 @@ Confirm the provider and model with `llma-evaluation-judge-models`.
 Call it with no arguments to see the whole catalog at once.
 Providers PostHog funds no models for come back empty unless you pass `key_id` for one of the team's keys; the response's `providers` list flags which ones those are.
 Prefer pinning the chosen key so a later team-wide active-key change does not change how the evaluation runs.
-Leave `provider_key_id` as `null` only after `llma-evaluation-config-get` confirms the active key is usable and its provider matches.
+Leave `provider_key_id` as `null` to run on the team's active key; before enabling, confirm with
+`llma-evaluation-config-get` that the active key is usable and its provider matches.
 
-If there is no usable key, you may still create a disabled draft for the user to review. Do not spot-run or
-enable it. Ask the user to add or validate a key in the UI before continuing.
+No usable key does not block creation. Create the disabled draft with a valid `provider` and `model` for
+the user to review. Do not spot-run or enable it, and ask the user to add or validate a key in the UI so it
+can be enabled later.
 
 ### 2.4 — Create it disabled
 
@@ -261,8 +265,9 @@ posthog:llma-evaluation-create
 ```
 
 For `llm_judge`, swap `evaluation_config` to `{ "prompt": "…" }` and add
-`"model_configuration": { "provider": "openai", "model": "gpt-5-mini", "provider_key_id": "<uuid of an ok-state key from llma-provider-key-list>" }`.
-Use `null` only when the active team key is `ok` and uses the same provider. Full field reference:
+`"model_configuration": { "provider": "openai", "model": "gpt-5-mini", "provider_key_id": null }`.
+Pin `provider_key_id` to the UUID of an `ok`-state key from `llma-provider-key-list` to run on one specific
+key; `null` runs on the team's active key. Either value saves a draft. Full field reference:
 [references/evaluation-payload.md](references/evaluation-payload.md).
 
 ### 2.5 — Verify the scope before enabling
@@ -283,7 +288,8 @@ For generation targets, `count()` is the run volume. For trace targets, count di
 `$ai_trace_id` values because matching generations from the same trace schedule only one run.
 
 If volume is high, set `rollout_percentage` below 100 to sample. Spot-check the evaluator with
-`llma-evaluation-test-hog` (hog) or `llma-evaluation-run` against one generation (llm_judge).
+`llma-evaluation-test-hog` (hog) or `llma-evaluation-run` against one generation (llm_judge; skip the
+spot-run for a judge with no usable key, per 2.3).
 Both tools currently use generation samples; for a trace target they can check shared source or prompt behavior,
 but they do not reproduce the complete settled trace. Review the first live trace results before increasing rollout.
 
@@ -354,8 +360,9 @@ creating so the user can review and toggle it in the UI.
   criterion genuinely can't be coded.
 - **Always create disabled, verify scope, then enable.** An eval firing on the wrong events is worse than
   none — noise, and (for llm_judge) cost.
-- **Configure llm_judge credentials before running.** A judge needs a valid provider and model plus a usable
-  provider key. `provider_key_id` may be `null` only when the matching active team key can be used.
+- **A judge draft needs only a provider and model.** A usable provider key is needed to enable it, not to
+  save it, so no `ok` key means keep it disabled — not stop. `provider_key_id` may be `null` to run on the
+  team's active key.
 - **`bytecode` is server-written** for hog evals — never pass it; send only `evaluation_config.source`.
 - For cluster-scoped evals, identify the cluster with `exploring-llm-clusters`, then translate its event
   filter into `conditions`.