Skip to content

feat(analytics): record how long a successful AI generation takes - #606

Open
Guotai812 wants to merge 1 commit into
mainfrom
feat/1632-ai-generation-duration
Open

Guotai812 wants to merge 1 commit into
mainfrom
feat/1632-ai-generation-duration

Conversation

@Guotai812

Copy link
Copy Markdown
Collaborator

Summary

  • New PostHog event ai_task_generation_succeeded, fired once per AI task generation turn that returns a result (ReceiveGenerationResult):
    • duration_ms: from mic release (voice) or the send tap (text) to the final result, timed on the phone.
    • transcript_ms (voice only): same start to ReceiveTranscript, to tell slow transcription from slow generation.
    • task_count: every draft shown after the turn (tasks + recurring tasks + notes), including drafts kept from earlier turns. Same rule as onboarding_voice_task_generated.task_count.
  • The voice clock now starts at mic release in useVoiceRecorder.stopAndUpload; before, it started after the audio file was read and base64-encoded. ai_task_generation_failed.duration_ms uses the same clock.
  • Code-review fixes in the same functions:
    • Voice send failures (AudioSubmitFailed) now carry duration_ms.
    • A voice network drop after the transcript arrived is labelled stage generation, not transcription.
    • A connection drop after a result or error already closed the request no longer records a second, failed outcome or shows an error toast.
  • Analytics impact: voice failure durations now include the recorder stop and upload prep, and some voice NetworkError failures move from transcription to generation. Compare old and new numbers by app version.
  • Mobile only, no backend change. After merge: check on TestFlight and Android alpha that voice and text still produce drafts (the recording hand-off changed).

Verified: local run on the iOS simulator against the local API. PostHog is off outside production, so the payloads were printed with a temporary log line (not committed):

voice                        ai_task_generation_succeeded {"input_mode":"voice","duration_ms":5776,"task_count":1,"transcript_ms":574}
text                         ai_task_generation_succeeded {"input_mode":"text","duration_ms":7857,"task_count":1}
voice, API stopped mid-run   ai_task_generation_failed {"input_mode":"voice","stage":"generation","error_code":"NetworkError","duration_ms":1975}
voice, API down              ai_task_generation_failed {"input_mode":"voice","stage":"send","error_code":"AudioSubmitFailed","duration_ms":19}

5 voice turns and 1 text turn each sent exactly one success event; the failed turns sent none. Phone totals line up with the server's model time (text turn: 7857 ms on the phone, 7783 ms for the model call). tsc 0 errors, expo lint no new warnings.

Release note

none

Status:

  • User-facing — announce it
  • Beta / partial — announce, but tagged as beta
  • Hidden in production (feature-flagged / not enabled for users) — don't announce
  • Internal only (refactor / infra / tests / CI / deps) — don't announce

- Send ai_task_generation_succeeded on every AI task generation turn that
  returns a result: input_mode, duration_ms, transcript_ms (voice only) and
  task_count.
- Time voice turns from mic release, so stopping the recorder and preparing
  the upload count toward the wait; ai_task_generation_failed uses the same
  clock.
- Voice send failures carry duration_ms, a voice drop after the transcript is
  labelled generation, and a drop after a request already closed is no longer
  recorded as a second outcome.
@Guotai812 Guotai812 added the L2 Review level 2: AI review + 1 tech-lead candidate label Oct 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Test Results

145 tests   145 ✅  11s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit 01729ac.

@github-actions

Copy link
Copy Markdown
Contributor
Package Line Rate Branch Rate Complexity Health
BlotzTask 4% 36% 2957 ❌
BlotzTask 4% 36% 2957 ❌
Summary 4% (5086 / 117230) 36% (1124 / 3166) 5914 ❌

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

L2 Review level 2: AI review + 1 tech-lead candidate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants