-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
fix(cloudflare): Enforce flush timeout across Workflow lifecycle #24483
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
matthewbjones
wants to merge
6
commits into
getsentry:develop
Choose a base branch
from
matthewbjones:feat/cloudflare-enforce-flush-timeout
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
643d751
fix(cloudflare): Enforce flush timeout across Workflow lifecycle
matthewbjones acd76c4
fix(cloudflare): Preserve transport budget during flush
matthewbjones 6dad74d
test(cloudflare): Stabilize flush timeout coverage
matthewbjones 48f22a8
Revert to make the diff easier for me
JPeer264 fb63820
fix(cloudflare): Abort pending sends when a flush times out
JPeer264 f0fc658
fixup! fix(cloudflare): Abort pending sends when a flush times out
JPeer264 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
52 changes: 52 additions & 0 deletions
52
dev-packages/cloudflare-integration-tests/suites/flush-timeout/index.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| import * as Sentry from '@sentry/cloudflare'; | ||
| import { WorkflowEntrypoint } from 'cloudflare:workers'; | ||
| import type { WorkflowEvent, WorkflowStep } from 'cloudflare:workers'; | ||
| import { lastSend } from './lastSend'; | ||
|
|
||
| interface Env { | ||
| SERVER_URL: string; | ||
| ISSUE_WORKFLOW: Workflow; | ||
| } | ||
|
|
||
| // The Workflow from https://github.com/getsentry/sentry-javascript/issues/24482. After its steps it flushes | ||
| // with the same timeout the SDK uses after each step and reports to SERVER_URL how the pending send ended. | ||
| export class IssueWorkflow extends WorkflowEntrypoint<Env> { | ||
| async run(_event: WorkflowEvent<unknown>, step: WorkflowStep): Promise<void> { | ||
| for (let index = 0; index < 100; index++) { | ||
| await step.do(`step-${index}`, async () => index); | ||
| } | ||
|
|
||
| // Locally the step spans are not sent before `run()` returns, so flush here with the timeout the SDK uses | ||
| // after each step, while the Workflow can still observe whether the pending send gets aborted. | ||
| lastSend.aborted = false; | ||
| const flushed = await Sentry.flush(2000); | ||
| const send = lastSend.aborted ? 'aborted' : 'not aborted'; | ||
| await fetch(`${this.env.SERVER_URL}/result`, { method: 'POST', body: JSON.stringify({ flushed, send }) }); | ||
| } | ||
| } | ||
|
|
||
| export default { | ||
| async fetch(request, env, ctx) { | ||
| const url = new URL(request.url); | ||
|
|
||
| if (url.pathname === '/workflow/trigger') { | ||
| const instance = await env.ISSUE_WORKFLOW.create(); | ||
| return Response.json({ id: instance.id }); | ||
| } | ||
|
|
||
| // The flush runs inside the invocation, so the send is still pending when its drain times out. | ||
| if (url.pathname === '/flush-with-timeout') { | ||
| Sentry.captureException(new Error('Captured on /flush-with-timeout')); | ||
| lastSend.aborted = false; | ||
| const flushed = await Sentry.flush(500); | ||
| return Response.json({ flushed, send: lastSend.aborted ? 'aborted' : 'not aborted' }); | ||
| } | ||
|
|
||
| if (url.pathname === '/pending-wait-until') { | ||
| ctx.waitUntil(new Promise(resolve => setTimeout(resolve, 120_000))); | ||
| Sentry.captureException(new Error('Captured on /pending-wait-until')); | ||
| } | ||
|
|
||
| return new Response('ok'); | ||
| }, | ||
| } satisfies ExportedHandler<Env>; |
25 changes: 25 additions & 0 deletions
25
dev-packages/cloudflare-integration-tests/suites/flush-timeout/instrument.server.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import { defineCloudflareOptions } from '@sentry/cloudflare'; | ||
| import { lastSend } from './lastSend'; | ||
|
|
||
| interface Env { | ||
| SENTRY_DSN: string; | ||
| SERVER_URL: string; | ||
| // "true" sends envelopes to SERVER_URL, a server that never answers | ||
| SLOW_INGEST?: string; | ||
| // "false" creates one client per invocation, which waits for the invocation's flush lock | ||
| CACHE_CLIENT?: string; | ||
| // "true" samples every trace, so the Workflow steps create spans to send | ||
| TRACING?: string; | ||
| } | ||
|
|
||
| export default defineCloudflareOptions((env: Env) => ({ | ||
| dsn: env.SLOW_INGEST === 'true' ? `${env.SERVER_URL.replace('://', '://public@')}/1337` : env.SENTRY_DSN, | ||
| cacheClient: env.CACHE_CLIENT !== 'false', | ||
| tracesSampleRate: env.TRACING === 'true' ? 1 : undefined, | ||
| transportOptions: { | ||
| fetch: (input, init) => { | ||
| init?.signal?.addEventListener('abort', () => (lastSend.aborted = true)); | ||
| return fetch(input, init); | ||
| }, | ||
| }, | ||
| })); |
2 changes: 2 additions & 0 deletions
2
dev-packages/cloudflare-integration-tests/suites/flush-timeout/lastSend.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| // Whether the transport aborted an envelope fetch since `aborted` was last reset. | ||
| export const lastSend: { aborted: boolean } = { aborted: false }; |
76 changes: 76 additions & 0 deletions
76
dev-packages/cloudflare-integration-tests/suites/flush-timeout/test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| import type { Envelope, Event } from '@sentry/core'; | ||
| import { createServer } from 'node:http'; | ||
| import type { AddressInfo } from 'node:net'; | ||
| import { expect, it, onTestFinished } from 'vitest'; | ||
| import { createRunner } from '../../runner'; | ||
|
|
||
| // Starts an ingest server that never answers envelope requests, so every send stays pending. The Workflow | ||
| // posts its result to `/result`, which resolves the returned promise with the posted body. | ||
| async function startSilentIngest(): Promise<{ url: string; result: Promise<unknown> }> { | ||
| let resolveResult!: (body: unknown) => void; | ||
| const result = new Promise(resolve => (resolveResult = resolve)); | ||
|
|
||
| const server = createServer((req, res) => { | ||
| if (req.url !== '/result') { | ||
| return; | ||
| } | ||
| let body = ''; | ||
| req.on('data', chunk => (body += chunk)); | ||
| req.on('end', () => { | ||
| res.end(); | ||
| resolveResult(JSON.parse(body)); | ||
| }); | ||
| }); | ||
| await new Promise<void>(resolve => server.listen(0, resolve)); | ||
| onTestFinished(() => { | ||
| server.closeAllConnections(); | ||
| server.close(); | ||
| }); | ||
|
|
||
| return { url: `http://localhost:${(server.address() as AddressInfo).port}`, result }; | ||
| } | ||
|
|
||
| it.for([true, false])( | ||
| 'cacheClient: %s - aborts a send that is still pending when the flush times out', | ||
| async (cacheClient, { signal }) => { | ||
| const ingest = await startSilentIngest(); | ||
|
|
||
| const runner = createRunner(__dirname) | ||
| .withServerUrl(ingest.url) | ||
| .withWranglerArgs('--var', 'SLOW_INGEST:true', '--var', `CACHE_CLIENT:${cacheClient}`) | ||
| .start(signal); | ||
|
|
||
| const result = await runner.makeRequest('get', '/flush-with-timeout'); | ||
| expect(result).toEqual({ flushed: false, send: 'aborted' }); | ||
| }, | ||
| ); | ||
|
|
||
| // A cached client sends the step spans in eager drains the Workflow cannot wait for, so this runs with one | ||
| // client per invocation. The transport abort itself is covered for both modes by the test above. | ||
| it('cacheClient: false - the Workflow from #24482 aborts its pending send when the flush times out', async ({ | ||
| signal, | ||
| }) => { | ||
| const ingest = await startSilentIngest(); | ||
|
|
||
| const runner = createRunner(__dirname) | ||
| .withServerUrl(ingest.url) | ||
| .withWranglerArgs('--var', 'SLOW_INGEST:true', '--var', 'CACHE_CLIENT:false', '--var', 'TRACING:true') | ||
| .start(signal); | ||
|
|
||
| await runner.makeRequest('get', '/workflow/trigger'); | ||
| expect(await ingest.result).toEqual({ flushed: false, send: 'aborted' }); | ||
| }); | ||
|
|
||
| it('cacheClient: false - delivers events while a user waitUntil task is still running', async ({ signal }) => { | ||
| const runner = createRunner(__dirname) | ||
| .withWranglerArgs('--var', 'CACHE_CLIENT:false') | ||
| .expect((envelope: Envelope) => { | ||
| const event = envelope[1]?.[0]?.[1] as Event; | ||
| expect(event.exception?.values?.[0]?.value).toBe('Captured on /pending-wait-until'); | ||
| }) | ||
| .unordered() | ||
| .start(signal); | ||
|
|
||
| await runner.makeRequest('get', '/pending-wait-until'); | ||
| await runner.completed(); | ||
| }); |
7 changes: 7 additions & 0 deletions
7
dev-packages/cloudflare-integration-tests/suites/flush-timeout/vite.config.mts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| import { cloudflare } from '@cloudflare/vite-plugin'; | ||
| import { sentryCloudflareVitePlugin } from '@sentry/cloudflare/vite'; | ||
| import { defineConfig } from 'vite'; | ||
|
|
||
| export default defineConfig({ | ||
| plugins: [cloudflare(), sentryCloudflareVitePlugin()], | ||
| }); |
14 changes: 14 additions & 0 deletions
14
dev-packages/cloudflare-integration-tests/suites/flush-timeout/wrangler.jsonc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| { | ||
| "$schema": "../../node_modules/wrangler/config-schema.json", | ||
| "name": "cloudflare-flush-timeout", | ||
| "main": "index.ts", | ||
| "compatibility_date": "2025-06-17", | ||
| "compatibility_flags": ["nodejs_compat"], | ||
| "workflows": [ | ||
| { | ||
| "name": "issue-workflow", | ||
| "binding": "ISSUE_WORKFLOW", | ||
| "class_name": "IssueWorkflow", | ||
| }, | ||
| ], | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.