Skip to content

feat(feature-flags): add wizard feature-flags install program - #1192

Draft
fristovic wants to merge 5 commits into
PostHog:mainfrom
fristovic:feat/feature-flags-program
Draft

feat(feature-flags): add wizard feature-flags install program#1192
fristovic wants to merge 5 commits into
PostHog:mainfrom
fristovic:feat/feature-flags-program

Conversation

@fristovic

@fristovic fristovic commented Sep 1, 2026

Copy link
Copy Markdown

Summary

Adds wizard feature-flags, a top-level install program (same shape as mcp-analytics) that runs the mill skill feature-flags-setup.

The gap: default wizard is product analytics, wizard audit feature-flags is read-only, wizard migrate is vendor porting. This is install + instrument on Next.js App Router only (one stack, one pattern). Sarah endorsed the top-level command.

What this PR wires:

  • Command, registry, bin.ts, abort cases, CLI/shape tests
  • OAuth feature_flag:read + feature_flag:write (not agent-skill’s property_definition:read)
  • Switchboard binding: linear / pi / GPT5_6_TERRA_MODEL / medium — still createSkillProgram, no orchestrator
  • Mill skill as the source of truth for steps; the agent prompt only points at it

What the paired mill skill does (PostHog/context-mill#378):

  • Server evaluateFlags() once per request → bootstrap into PostHogProvider → CI /flags polling off
  • One wizard_ask: skip first (install only, no new flag) or confirm an additive UI path
  • On confirm: create one boolean flag at 0% rollout (never 100%) and gate that path. Production users stay on current UI until someone raises rollout in PostHog
  • Abort if flag create fails

Depends on mill #378.

Test plan

Unit (this PR)

  • pnpm test (at least feature-flags.test.ts, program-scopes.test.ts, programs-cli.test.ts, switchboard.test.ts, flags.test.ts)
  • pnpm try feature-flags --help shows the new command and does not collide with wizard audit feature-flags
  • Unflagged feature-flags resolves to linear + pi + terra medium (not DEFAULT_BINDING)

E2E (needs mill #378 served locally)

pnpm try feature-flags --local-context-mill --install-dir /tmp/wizard-flags-pr --no-telemetry
  • Re-auth if this token predates the new scopes — consent must include feature flag read + write. This program should not ask for property_definition:read.
  • Intro TUI: “Let’s run the feature-flags-setup skill”
  • Skip path: “Skip gating — install only”. Diff has evaluateFlags + provider + env; no new flag in PostHog; layout does not isEnabled an invented key.
  • Confirm path: on a second fresh copy, pick the recommended additive target (not skip).
    • Flag exists, boolean, active, 0% rollout
    • One additive gated component (useFeatureFlagEnabled(..., false))
    • pnpm dev — UI matches today (flag off)
    • PostHog → that flag → rollout 100% → save → reload — extra UI appears
    • Rollout 0% → reload — extra UI gone
    • ./posthog-feature-flags-report.md documents 0% and the kill-switch steps
  • Non-App Router app → abort “unsupported stack for feature flags”, no file edits
  • Distinct from npx @posthog/wizard audit feature-flags (audit still read-only)

Cross-repo CI once mill is up: /wizard-ci all wizard:feat/feature-flags-program on mill #378.

Made with Cursor


Docs: PostHog/posthog.com#19911 (/docs/feature-flags/installation/ai-wizard). Local docsUrl in src/lib/programs/feature-flags/index.ts now points at that page. Include the one-liner when you commit Learn cards. Do not un-draft this PR from the docs track.

Comment thread src/lib/programs/feature-flags/index.ts Outdated
'or run `npx @posthog/wizard` for a general PostHog install.',
},
{
match: /^could not locate a UI surface to gate$/i,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd consider maybe degrading rather than stopping the run

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call! I'll see what I can do.

'mcp-remove': DEFAULT_BINDING,
'mcp-tutorial': DEFAULT_BINDING,
'mcp-analytics': DEFAULT_BINDING,
'feature-flags': {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love to see evals if you have any for why this model! it's okay if it's just personal preference too :)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually Tried Vincent’s Slack hint out of curiosity. I don't have any real evals I just wanted to see another model appear in the logs 🤣 I’ll revert to default Sonnet unless you’d rather keep the experiment.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need! was just curious :)

@fristovic

Copy link
Copy Markdown
Author

Official docs page (draft): PostHog/posthog.com#19911. docsUrl in the local working copy of src/lib/programs/feature-flags/index.ts now points at https://posthog.com/docs/feature-flags/installation/ai-wizard. Include that one-liner when you commit Learn cards.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

your personality really comes through the Deck (big fan of putting personality in it!!), and I'm glad you played with the Primitives and added some visuals. visually, it's nice.

on the content itself:

  • I came away from watching the full deck not fully understanding what a feature flag is (I wore my junior engineering hat for this one). I think it's worth naming it upfront, even if it's a simple definition to build concretely on top of. everything as is, feels abstract.
  • the learn deck jumps between example apps that don't have a relationship to one another (or if they do, I was struggling to find a relationship) so it felt like I couldn't anchor to anything

TL;DR I think there's just some assumption here that the user is familiar with flags and has them running at scale

I'd maybe cut it to orient a user as if they weren't 100% familiar with flags, with the intention that if they watch the Learn cards, they'll leave feeling like they at least grasp the concept

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants