Skip to content

feat: include rationale in browserless_agent analytics event#147

Open
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1782506276-agent-analytics-rationale
Open

feat: include rationale in browserless_agent analytics event#147
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1782506276-agent-analytics-rationale

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jun 26, 2026

Copy link
Copy Markdown

Summary

  • Adds rationale: params.rationale ?? null to the sendAnalytics closure in src/tools/agent.ts, so every MCP Tool Request event for browserless_agent now includes the LLM-supplied intent string.

Changes

In src/tools/agent.ts (the sendAnalytics closure ~line 367), the properties object passed to analytics?.fireToolRequest(token, 'browserless_agent', {...}) now includes:

rationale: params.rationale ?? null,

rationale is a top-level optional string on AgentParamsSchema (defined in src/tools/schemas.ts:700-716) — a ≤50-char present-continuous label the LLM provides per call (e.g. "Logging in", "Filling the search form").

Privacy note: event_properties are sent to SQS (see src/lib/analytics.ts). This change now ships the free-text rationale string — which is LLM-generated intent, not raw end-user conversation — through the same pipeline. The field is capped at 50 chars by schema description and contains only action summaries, so the added PII exposure is minimal.

Test plan

  • npm test passes locally (439 passing)
  • npm run build passes locally (type-checks clean)
  • N/A — no new dependencies, no runtime behaviour change beyond analytics payload

Checklist

  • My commits follow the conventional-commit prefix convention
  • I have not introduced new dependencies without flagging them in the summary

Link to Devin session: https://app.devin.ai/sessions/679a080a73264dda9026d574edb4874c
Requested by: @xsvfat


Open in Devin Review

Summary by CodeRabbit

  • Bug Fixes
    • Analytics events now include the reasoning field when available, improving completeness of recorded usage data.

Co-Authored-By: Sean Yang <sean@browserless.io>
@xsvfat xsvfat self-assigned this Jun 26, 2026
@devin-ai-integration

Copy link
Copy Markdown
Author
Original prompt from Sean Yang

In the browserless/browserless-mcp repository, capture the user-intent string in analytics for the browserless_agent tool.

Context: The MCP server only receives structured tool arguments, not the raw end-user conversation. The browserless_agent tool already requires a rationale parameter (defined in src/tools/schemas.ts around lines 710-726) — a short human-readable reason for each call — but it is currently NOT included in the analytics event.

Change: In src/tools/agent.ts, the sendAnalytics closure (around lines 412-424) builds the properties passed to analytics?.fireToolRequest(token, 'browserless_agent', {...}). Add the rationale to these properties, e.g. rationale: params.rationale ?? null (use the actual field name/path from AgentParamsSchema; if rationale lives per-command, decide whether to join multiple command rationales).

Verify the field name and shape by inspecting AgentParamsSchema and the rationale definition in src/tools/schemas.ts first.

Expected outcome: every MCP Tool Request event for browserless_agent now includes a rationale property containing the LLM-supplied intent string, giving visibility into what the user asked the agent to do.

Note in the PR description the privacy consideration: event_properties are sent to SQS (see src/lib/analytics.ts), so this now ships free-text user-intent strings; confirm this is acceptable for the analytics pipeline.

@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@devin-ai-integration[bot], we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 57 minutes and 28 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 96a4ede1-2449-4ce0-92d1-1653817e80dd

📥 Commits

Reviewing files that changed from the base of the PR and between e155bfa and 933f0f8.

📒 Files selected for processing (1)
  • src/tools/agent.ts

Walkthrough

The browserless_agent tool now includes params.rationale in the analytics payload sent by sendAnalytics, using null when the value is absent.

Changes

Browserless agent analytics

Layer / File(s) Summary
Include rationale in analytics payload
src/tools/agent.ts
The browserless_agent analytics properties object now includes rationale: params.rationale ?? null.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Poem

A bunny hops through data trails,
With rationale tucked in tiny rails.
Null when quiet, clear when new,
Analytics now knows the cue.
sniff sniff 🐰

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding rationale to the browserless_agent analytics event.
Description check ✅ Passed The description follows the template closely with summary, changes, test plan, and checklist; only related issues is omitted.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch devin/1782506276-agent-analytics-rationale

Comment @coderabbitai help to get the list of available commands.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/tools/agent.ts`:
- Line 379: The `rationale` value is being forwarded from `fireToolRequest(...)`
into analytics without enforcement, so validate/redact it before use. Update
`AgentParamsSchema` and the `fireToolRequest` path to ensure `params.rationale`
is truncated and stripped of URLs/credentials before assigning it, while still
allowing null when absent. Use the existing `rationale` field in
`AgentParamsSchema` and the `fireToolRequest` call site as the main places to
apply the fix.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4284759d-87a8-46d9-9aa6-b59efe9f95bd

📥 Commits

Reviewing files that changed from the base of the PR and between 93d9189 and e155bfa.

📒 Files selected for processing (1)
  • src/tools/agent.ts

Comment thread src/tools/agent.ts Outdated
Co-Authored-By: Sean Yang <sean@browserless.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant