Skip to content

[Suggestion] OpenAI Agents SDK with Deepgram STT and TTS as tool providers (Python) #288

Description

@deepgram-robot

What to build

A working example integrating Deepgram's STT and TTS APIs as tool providers within the OpenAI Agents SDK. The agent should be able to transcribe audio files/streams and generate speech from text as part of its tool-calling workflow — for example, an agent that listens to a voice message, transcribes it, reasons about the content, and responds with synthesized speech.

Why this matters

The OpenAI Agents SDK is becoming a standard for building tool-using AI agents, and developers need voice capabilities in their agent pipelines. There's no existing example showing Deepgram as a tool provider in this framework. By providing a reference implementation, developers building OpenAI Agents SDK projects can add audio transcription and speech synthesis without evaluating alternatives — Deepgram becomes the default voice layer.

Suggested scope

  • Language: Python
  • Framework: OpenAI Agents SDK (latest)
  • Deepgram APIs: Pre-recorded STT (Nova-3), TTS (Aura), optionally streaming STT
  • Tools: transcribe_audio tool (file/URL → transcript), speak_text tool (text → audio file)
  • Demo scenario: Voice message assistant that transcribes incoming audio, extracts key information, and responds with spoken audio
  • Complexity: Intermediate — demonstrates tool definition, registration, and agent orchestration

Acceptance criteria

  • Runnable with minimal setup (clone, add API keys, run)
  • README explains the pattern clearly
  • Uses current Deepgram SDK version and OpenAI Agents SDK
  • Tools are properly typed with Pydantic models
  • Includes sample audio for testing the transcription tool
  • Shows both STT and TTS integration in a single agent workflow

Raised by the DX intelligence system.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions