Skip to content

feat(ai): list_maple_feedback, the Maple team's cross-org reader for agent feedback - #2330

Merged
Makisuo merged 3 commits into
mainfrom
feat/agent-feedback-triage
Nov 7, 2026
Merged

Makisuo merged 3 commits into
mainfrom
feat/agent-feedback-triage

Conversation

@Makisuo

@Makisuo Makisuo commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Agents send feedback about Maple through send_maple_feedback and POST /v2/agent_feedback into the agent_feedback table. Nothing let the Maple team read it across orgs: GET /v2/agent_feedback lists only the caller's own org, and traces record only argument names. 23 prod reports since 10-25 have never been read.

What

  • AgentFeedbackService.triage(callerOrgId, filter): a cross-org read (.crossTenant()), newest first, with filters for kind, impact, start time, related_to (case-insensitive substring, LIKE-escaped), org and limit (max 200). It refuses any caller org except MAPLE_INTERNAL_ORG_ID, and it refuses everyone while that variable is unset (self-hosted, misconfigured). The check runs before the database is touched.
  • New MCP tool list_maple_feedback (read-only). It renders a table plus each report's full reason and details. It also refuses callers whose audit actor is system or missing, which covers Maple's own chat agents and the bot (they use the internal service token) and in-process calls. That way customer-written feedback text never reaches the bot's channels or the context of internal agents that hold write tools. Only a person (OAuth) or API key in Maple's org, on the public /mcp transport, can read it.
  • MAPLE_INTERNAL_ORG_ID (already used by ingest, and set in Infisical prod and dev) is now bound on maple-ai as an optional plain var and added to Env.

Owned by another session

  • apps/ai/src/mcp/tools/registry.ts (M3): only the import line and the registration line.
  • The tool list is static, so list_maple_feedback is listed for every MCP client even though it refuses them. Hiding it per org (or adding an mcp-only audience in tools/types.ts) belongs to M3/M2. Proposed as a follow-up.

Verification

  • Service tests (PGlite, AgentFeedbackService.test.ts, 6/6): the internal org reads every org newest first; each filter narrows, including a decoy row proving _ is not a wildcard; any other org is refused, including one that has feedback of its own; every org is refused while the variable is unset.
  • Tool tests through the registry (list-maple-feedback.test.ts): the internal org with an OAuth actor gets both orgs' rows and the full text. These callers are refused with insufficient_scope: a customer org (even when filtering to itself), the internal org with a system actor (agents and the bot), the internal org with no actor, and any org while the variable is unset.
  • apps/ai tools suite: 493/493. Typecheck passes for apps/ai, apps/api, packages/backend and packages/domain. oxlint -c .oxlintrc.effect.json reports nothing on the changed files. Tool descriptors regenerated.

The triage summary of the existing prod reports will be added once this is deployed.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

Summary by CodeRabbit

  • New Features
    • Added a feedback-report listing tool with filters for feedback type, impact, time, related text, and organization.
    • Authorized internal callers can view reports across organizations; results are sorted newest first and include report details.
    • Added clear responses for empty results and when the result limit is reached.

…agent feedback

Agents file feedback through send_maple_feedback and POST /v2/agent_feedback, but
nothing let the Maple team read it across orgs. AgentFeedbackService.triage reads
every org's reports and refuses any caller org but MAPLE_INTERNAL_ORG_ID (fails
closed when unset). The MCP tool also refuses Maple's own agents and the bot, so
customer feedback text only reaches a person or API key on the /mcp transport.
@maple-review-bot

This comment has been minimized.

@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The change adds a feedback triage service restricted to the configured internal organization and exposes it through a registered MCP tool. The tool supports filters, returns report data, and renders matching feedback.

Changes: Feedback triage service

Layer / File(s) Summary
Triage authorization and query
packages/domain/src/http/agent-feedback.ts, packages/backend/src/platform/Env.ts, apps/ai/src/worker.ts, packages/backend/src/services/feedback/AgentFeedbackService.ts, packages/backend/src/services/feedback/AgentFeedbackService.test.ts
The environment config adds MAPLE_INTERNAL_ORG_ID. The service adds a triage operation that checks the caller organization and queries feedback with optional filters. Service tests cover authorization and filtering.
MCP report output contract
packages/domain/src/mcp-outputs/feedback.ts, packages/domain/src/mcp-outputs/catalog.ts
The output schemas describe feedback reports and list results. The catalog maps list_maple_feedback to its output schema.
MCP tool execution and registration
apps/ai/src/mcp/tools/list-maple-feedback.ts, apps/ai/src/mcp/tools/registry.ts, apps/ai/src/mcp/tools/__tests__/list-maple-feedback.test.ts
The tool validates inputs, calls triage, maps results and errors, and renders reports. The registry adds the tool, and integration tests cover successful and refused calls.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Caller as MCP caller
  participant Tool as list_maple_feedback
  participant Service as AgentFeedbackService
  participant Database as Feedback database
  Caller->>Tool: Submit filters
  Tool->>Service: Call triage with tenant and filters
  Service->>Service: Check configured internal organization
  Service->>Database: Query filtered feedback
  Database-->>Service: Return matching records
  Service-->>Tool: Return triage results
  Tool-->>Caller: Return report output
Loading

Possibly related PRs

Suggested reviewers: jeremyfunk

Merge Risk: Minimal · up to 7f2c6

This adds a read-only internal tool for reading agent feedback. Access is restricted to Maple's internal organization and refused otherwise. No merge-blocking risk was found; the positive cross-org read is to be confirmed in production after merge.

Pre-merge checks: Passed 5

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly identifies the new cross-organization agent feedback reader.
Docstring Coverage Passed No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 10 files.

Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 57154e48-65ae-4706-8789-d84bc8746bd9
Commits

Reviewing files that changed from the base of the PR and between 0e001d7 and 7f2c60e.

Files ignored due to path filters: 1

  • apps/ai/src/mcp/tool-descriptors.generated.ts is excluded by !**/*.generated.*
Files selected for processing: 10

  • apps/ai/src/mcp/tools/__tests__/list-maple-feedback.test.ts
  • apps/ai/src/mcp/tools/list-maple-feedback.ts
  • apps/ai/src/mcp/tools/registry.ts
  • apps/ai/src/worker.ts
  • packages/backend/src/platform/Env.ts
  • packages/backend/src/services/feedback/AgentFeedbackService.test.ts
  • packages/backend/src/services/feedback/AgentFeedbackService.ts
  • packages/domain/src/http/agent-feedback.ts
  • packages/domain/src/mcp-outputs/catalog.ts
  • packages/domain/src/mcp-outputs/feedback.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

@Makisuo Makisuo added the preview:web Deploy a Workers-only PR preview (no ingest/Electric); see docs/pr-previews.md label Nov 7, 2026
@Makisuo

Makisuo commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

Manual check done: MAPLE_INTERNAL_ORG_ID is set in Infisical prod and dev (the same value ingest already requires at deploy, Maple's internal org, which is also the org the maple CLI uses). maple-ai now binds it through optionalPlain in apps/ai/src/worker.ts, so the root deploy passes it the same way it does for ingest. The McpQueryError issues listed against registry.ts belong to M3's session; this PR only adds the import and the registration line.

@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

🍁 Maple PR preview

Note

Preview resources were removed when the preview label came off.

Final commit eb4bc79 · View workflow run

@Makisuo

Makisuo commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

Preview check (pr-2330)

A separate verification agent tested https://api-pr-2330.maple.dev/mcp with a fresh full-access API key in the Preview org (org_3KkfPhr7sdlpq6iVXJhuATABBtb). The key was revoked afterwards.

Check Result
tools/list includes list_maple_feedback and send_maple_feedback PASS (127 tools)
send_maple_feedback writes a row PASS (afb_qJg6oBw4YC2hdZtq4AQYWD)
list_maple_feedback with {}, {"org_id": "<own org>"} and {"since": "2026-11-01 00:00:00"} PASS: all three refused, no reports returned

Refusal text: Authentication failed: Reading agent feedback across organizations is limited to the Maple team. The "Authentication failed:" prefix comes from how the dispatcher renders McpAuthInvalidError, which is M3's file. A caller who is authenticated but not permitted should get wording like "Not permitted"; noted as a follow-up for M3.

The preview has no data from the internal org, so the positive read (internal org gets every org's reports) is covered by the tests here and gets checked in prod after merge.

Traces (deployment.environment = pr-2330)

Span Count p50 p95 Errors
tools/call list_maple_feedback 3 117 ms 140 ms 0
McpTool.listMapleFeedback 3 0 ms 0 ms 0
AgentFeedbackService.triage 3 0 ms 0 ms 3 (expected refusals)
tools/call send_maple_feedback 1 188 ms 0

Span tree of one refused call (trace 5e0b48d27195e4b3d03a627d618818fa): ApiKeysService.resolveByBearer 9 ms, then McpToolRateLimiter.check, then McpTool.listMapleFeedback, where AgentFeedbackService.triage returns Error in 0 ms with no database span (refused before the query, as intended). The remaining 62 ms is the executor's existing AuditLogService.record warehouse insert, which runs on every tool call. No N+1s, no retries, no slow spans from this change.

@Makisuo Makisuo removed the preview:web Deploy a Workers-only PR preview (no ingest/Electric); see docs/pr-previews.md label Nov 7, 2026
…riage

# Conflicts:
#	apps/ai/src/mcp/tool-descriptors.generated.ts
@maple-review-bot

This comment has been minimized.

…riage

# Conflicts:
#	apps/ai/src/mcp/tool-descriptors.generated.ts
@Makisuo

Makisuo commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

Merged main (7f2c60e). The only conflict was the generated tool-descriptors.generated.ts; I regenerated it, so it is main's file plus the list_maple_feedback line.

#2334 resolves the "Authentication failed:" follow-up from the preview comment: the dispatcher now renders an insufficient_scope refusal as Permission denied: …, which is what this tool returns. The tool is read-only, so the new rule that hides write tools from read-only keys does not apply to it, and its org and caller checks are unchanged. After the merge, typecheck (ai, backend, domain) is clean, 73/73 tests pass (feedback service, the tool, the registry contract, registry, dispatcher and app tests), and lint is clean.

@maple-review-bot

maple-review-bot Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 10/10 · safe to merge. Cross-org read gated by MAPLE_INTERNAL_ORG_ID and the transport's credential class; both paths are pinned by the new service and registry tests.
no findings · tests covered · risk high · 2/2 new units observable

Adds AgentFeedbackService.triage, a cross-org newest-first read gated on MAPLE_INTERNAL_ORG_ID and on a person/API-key audit actor, and the list_maple_feedback MCP tool over it. The gate, LIKE escaping and filters are covered by tests; safe to merge. It does not touch the open McpQueryError issues: the registry diff is one import and one registration line, so those stacks are unchanged.

  • AgentFeedbackService.triage reads every org via .crossTenant() after the internal-org check
  • list_maple_feedback refuses a system or absent audit actor and maps the forbidden error to insufficient_scope
  • MAPLE_INTERNAL_ORG_ID bound on maple-ai as Env.MAPLE_INTERNAL_ORG_ID
  • ListMapleFeedbackOutput added to McpToolOutputs and the registry

Before merge

  • Manual · Confirm MAPLE_INTERNAL_ORG_ID in the maple-ai plain-var store for prd and dev, exactly equal to the internal org's tenant id

Production impact

Open errors in the changed files (2)
Issue Service Occurrences File
@maple/mcp/errors/McpQueryError (5 issues) maple-chat 33 apps/ai/src/mcp/tools/registry.ts
@maple/mcp/errors/McpQueryError (5 issues) maple-ai 15 apps/ai/src/mcp/tools/registry.ts

After this merges, Maple checks whether they stop.

7f2c60e · What was checked, coverage and telemetry details are on the Maple / review check. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

@Makisuo
Makisuo merged commit 1a220f1 into main Nov 7, 2026
38 checks passed
@Makisuo
Makisuo deleted the feat/agent-feedback-triage branch November 7, 2026 14:36
@maple-review-bot

Copy link
Copy Markdown

Merged with 1 step from "Before merge" still open. If it is done, tick it on the review comment; if not, now is the time.

  • Manual · Confirm MAPLE_INTERNAL_ORG_ID in the maple-ai plain-var store for prd and dev, exactly equal to the internal org's tenant id

@maple-review-bot

Copy link
Copy Markdown

Maple: after this shipped

✅ Shipped clean. Nothing this change touches got worse in the hour after it deployed.

Deployed to maple-api (production) at 2026-11-07 14:50 UTC as 1a220f1. Compared the 60 minutes before and after.

Errors in the changed files:

  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
Operation Calls/h Errors p95
POST /mcp 960 → 848 0.0% → 0.0% 764 → 1115 ms
POST /mcp 959 → 847 0.0% → 0.0% 1601 → 1734 ms
GET /mcp 37 → 1 0.0% → 0.0% 1005 → 45 ms
GET /mcp 37 → 1 0.0% → 0.0% 72 → 18 ms
POST /mcp 12 → 3 0.0% → 0.0% 10695 → 4233 ms

Maple looks once, an hour after the first deploy that carries the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant