-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
docs(javascript): Document TypeSafe and Vercel AI Jev tracing #19692
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+204
−0
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
72003a9
docs(javascript): Document TypeSafe and Vercel AI Jev tracing
sentry-junior[bot] 3da1b14
Apply suggestion from @andreiborza
andreiborza ef60334
docs(javascript): Remove TypeSafe integration card description
sentry-junior[bot] 3b03864
docs(javascript): Simplify Jev integration cross-reference
sentry-junior[bot] 9720b2d
docs(javascript): Align TypeSafe reference with Mistral integration
sentry-junior[bot] f96cdde
fix(docs): Register TypeSafe AI platform icon
sentry-junior[bot] 50f52f0
docs(javascript): Use TypeSafe AI display name
sentry-junior[bot] 38f5b75
docs(javascript): Move TypeSafe AI below Together AI
sentry-junior[bot] dbec94b
docs(javascript): Restore TypeSafe display name
sentry-junior[bot] bb85304
Merge branch 'master' into docs/typesafe-vercel-jev
andreiborza 1e77163
docs(javascript): List TypeSafe in server integration tables
sentry-junior[bot] 63ab76f
docs(javascript): List TypeSafe in Deno integration table
sentry-junior[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
151 changes: 151 additions & 0 deletions
151
docs/platforms/javascript/common/configuration/integrations/typesafe.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| --- | ||
| title: TypeSafe | ||
| description: "Adds instrumentation for TypeSafe Jev evaluations. (default)" | ||
| supportedCategories: | ||
| - server | ||
| - serverless | ||
| notSupported: | ||
| - javascript.cloudflare | ||
| - javascript.mastra | ||
| --- | ||
|
|
||
| <Alert> | ||
|
|
||
| Requires Sentry SDK version `11.1.0` or higher. | ||
|
|
||
| </Alert> | ||
|
|
||
| _Import name: `Sentry.typesafeIntegration`_ | ||
|
|
||
| This integration is enabled by default when <PlatformLink to="/tracing/">tracing</PlatformLink> is enabled. To customize it, see [Modifying Default Integrations](./../#modifying-default-integrations). | ||
|
|
||
| The `typesafeIntegration` adds instrumentation for [`@typesafe-ai/sdk`](https://www.npmjs.com/package/@typesafe-ai/sdk) to capture <PlatformLink to="/agent-tracing/">agent tracing</PlatformLink> `gen_ai.evaluate` spans, recording model, token usage, latency, and (when enabled) evaluation inputs and answers. | ||
|
|
||
| If you call Jev through Vercel AI SDK's `experimental_evaluate()`, see the [Vercel AI integration](/platforms/javascript/guides/node/configuration/integrations/vercelai/#trace-jev-evaluations) instead. | ||
|
|
||
| ```javascript | ||
| Sentry.init({ | ||
| dsn: "___PUBLIC_DSN___", | ||
| // Tracing must be enabled for agent tracing to work | ||
| tracesSampleRate: 1.0, | ||
| integrations: [Sentry.typesafeIntegration()], | ||
| }); | ||
| ``` | ||
|
|
||
| <PlatformSection supported={["javascript.nextjs"]}> | ||
|
|
||
| Run TypeSafe calls in the Node.js runtime. This automatic instrumentation isn't available in the Edge runtime. | ||
|
|
||
| </PlatformSection> | ||
|
|
||
| <PlatformSection supported={["javascript.deno"]}> | ||
|
|
||
| On Deno, preload Sentry's import hook so the TypeSafe SDK can be instrumented: | ||
|
|
||
| ```bash | ||
| deno run --allow-net --allow-env --allow-read --allow-sys --allow-write --preload=npm:@sentry/deno/import app.ts | ||
| ``` | ||
|
|
||
| </PlatformSection> | ||
|
|
||
| <PlatformSection supported={["javascript.bun", "javascript.elysia"]}> | ||
|
|
||
| On Bun, automatic instrumentation requires a build with `sentryBunPlugin()` and `@typesafe-ai/sdk` included in the bundle: | ||
|
|
||
| ```typescript {filename:build.ts} | ||
| import { sentryBunPlugin } from "@sentry/bun/plugin"; | ||
|
|
||
| await Bun.build({ | ||
| entrypoints: ["./src/index.ts"], | ||
| outdir: "./dist", | ||
| target: "bun", | ||
| plugins: [sentryBunPlugin()], | ||
| }); | ||
| ``` | ||
|
|
||
| For unbundled applications started with `bun run`, use [Manual Instrumentation](#manual-instrumentation). | ||
|
|
||
| </PlatformSection> | ||
|
|
||
| ## Supported Operations | ||
|
|
||
| Tracing support is added to `TypeSafeClient.systemOne()` calls to evaluate state against a set of questions using Jev. Each call creates a `gen_ai.evaluate` span. Recorded answers include probabilities and confidence values when returned by TypeSafe. | ||
|
|
||
| ## Supported Versions | ||
|
|
||
| - `@typesafe-ai/sdk`: `>=0.5.0 <1` | ||
|
|
||
| ## Options | ||
|
|
||
| Evaluation state and answers can contain sensitive data. Input and output recording are enabled by default. To disable them for TypeSafe, configure the integration explicitly: | ||
|
|
||
| ```javascript | ||
| Sentry.init({ | ||
| dsn: "___PUBLIC_DSN___", | ||
| tracesSampleRate: 1.0, | ||
| integrations: [ | ||
| Sentry.typesafeIntegration({ | ||
| recordInputs: false, | ||
| recordOutputs: false, | ||
| }), | ||
| ], | ||
| }); | ||
| ``` | ||
|
|
||
| ### `recordInputs` | ||
|
|
||
| _Type: `boolean` (optional)_ | ||
|
|
||
| Records evaluation `state` and `questions`. Defaults to `dataCollection.genAI.inputs`, which is `true` by default. | ||
|
|
||
| ### `recordOutputs` | ||
|
|
||
| _Type: `boolean` (optional)_ | ||
|
|
||
| Records evaluation `answers`. Defaults to `dataCollection.genAI.outputs`, which is `true` by default. | ||
|
|
||
| <PlatformSection notSupported={["javascript.deno", "javascript.solidstart"]}> | ||
|
|
||
| ## Manual Instrumentation | ||
|
|
||
| _Import name: `Sentry.instrumentTypeSafeClient`_ | ||
|
|
||
| If your application does not use automatic instrumentation, wrap your TypeSafe client after setting up Sentry. Use the returned client for your `systemOne()` calls. | ||
|
|
||
| <PlatformSection supported={["javascript.hono"]}> | ||
|
|
||
| Import Sentry from the entry point for your runtime: `@sentry/hono/node`, `@sentry/hono/bun`, or `@sentry/hono/cloudflare`. | ||
|
|
||
| </PlatformSection> | ||
|
|
||
| <PlatformSection notSupported={["javascript.hono"]}> | ||
|
|
||
| ```javascript | ||
| import * as Sentry from "___SDK_PACKAGE___"; | ||
| ``` | ||
|
|
||
| </PlatformSection> | ||
|
|
||
| ```javascript | ||
| import { TypeSafeClient, choice } from "@typesafe-ai/sdk"; | ||
|
|
||
| const typesafe = new TypeSafeClient({ apiKey: "your-typesafe-api-key" }); | ||
| const client = Sentry.instrumentTypeSafeClient(typesafe, { | ||
| recordInputs: false, | ||
| recordOutputs: false, | ||
| }); | ||
|
|
||
| const result = await client.systemOne({ | ||
| state: "I was charged twice. Please refund the extra charge.", | ||
| questions: { | ||
| department: choice("Which team should handle this?", { | ||
| billing: "Charges and refunds", | ||
| technical: "Bugs and outages", | ||
| }), | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| Use one instrumentation approach for each client. If automatic instrumentation is already active, you don't need to wrap the client. | ||
|
|
||
| </PlatformSection> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.