Audio Embedding API

A FastAPI-based API for generating vector embeddings from audio files using the resemblyzer library.

Features

• Upload audio files (WAV format recommended)
• Generate vector embeddings using resemblyzer's VoiceEncoder
• Return embeddings in a format suitable for storage in vector databases like Pinecone

Installation

1. Clone this repository
2. Install the required dependencies:

pip install -r requirements.txt

Usage

Starting the API server

Run the following command from the project directory:

python main.py

This will start the API server at http://0.0.0.0:8000.

API Endpoints

POST /generate-embedding/

Generates a vector embedding from an uploaded audio file.

Request:

• Method: POST
• Content-Type: multipart/form-data
• Body: audio_file (file)

Response:

{
  "filename": "example.wav",
  "embedding_dimension": 256,
  "embedding": [0.1, 0.2, ..., 0.3]
}

Example Usage with cURL

curl -X POST "http://localhost:8000/generate-embedding/" \
  -H "accept: application/json" \
  -H "Content-Type: multipart/form-data" \
  -F "audio_file=@/path/to/your/audio/file.wav"

vaughnnze@Vaughns-MacBook-Pro Downloads % curl -X POST \
  -F 'audio_file=@"Audio Clip (2025-07-08 14_00_31).m4a";type=audio/m4a' \
  http://localhost:8000/generate-embedding/

Example Usage with Python

import requests

url = "http://localhost:8000/generate-embedding/"
files = {"audio_file": open("path/to/your/audio/file.wav", "rb")}

response = requests.post(url, files=files)
embedding_data = response.json()

# Now you can use the embedding with a vector database like Pinecone
embedding = embedding_data["embedding"]

Integration with Vector Databases

The embeddings generated by this API can be stored in vector databases like Pinecone for similarity search and other operations.

Example with Pinecone

import pinecone
import requests

# Initialize Pinecone
pinecone.init(api_key="your-api-key", environment="your-environment")
index = pinecone.Index("your-index-name")

# Get embedding from API
url = "http://localhost:8000/generate-embedding/"
files = {"audio_file": open("path/to/your/audio/file.wav", "rb")}
response = requests.post(url, files=files)
embedding_data = response.json()

# Store in Pinecone
index.upsert(
    vectors=[
        {
            "id": "audio_1",  # Unique ID for this audio
            "values": embedding_data["embedding"]
        }
    ]
)

Notes

• The API works best with WAV format audio files
• The embedding dimension is 256 by default (determined by resemblyzer)

TODO

• Add authentication, rate limiting, logs and unit test coverage