Skip to content

README.md

Latest commit

 

History

History
182 lines (131 loc) · 5.69 KB

README.md

File metadata and controls

182 lines (131 loc) · 5.69 KB
Raw

SQLGraph

SQLGraph generates SQL Ontologies from relational databases and flat files to serve as the semantic foundation for GraphRAG systems. Based on the insight that GraphRAG is fundamentally a modeling problem rather than a database problem, SQLGraph represents concepts and relationships as explicit entities in a model that both humans and LLMs can reliably interpret and query.

The system is built around SQLAlchemy for database abstraction, Agno for agent orchestration, and an OpenAI-compatible API layer for LLM integration.

Core Concept

Traditional property graphs often embed meaningful concepts inside relationship properties. For example:

(Person)-[:EMPLOYED_AT {start_date: 2020, salary: 100000}]->(Company)

Here the concept of Employment is implicit and hidden inside an edge attribute. When an LLM attempts to generate queries over this structure, it must infer which relationships contain which attributes. This fragility breaks at scale.

SQLGraph follows an ontology-driven approach where relationships that carry meaning are promoted to first-class entities:

Person
Company
Employment
  - employee -> Person
  - employer -> Company
  - start_date
  - end_date
  - salary

The schema is explicit, transparent, and natively queryable in SQL -- the query language LLMs understand best.

Architecture

Module Responsibility
sqlgraph.db Database connectivity with automatic driver installation, schema discovery via SQLAlchemy Inspector
sqlgraph.ontology Ontology models, builder (from schema or DataFrame), exporter (JSON, SQL DDL, Markdown)
sqlgraph.llm OpenAI-compatible client for chat completion and SQL generation
sqlgraph.agents Agno-based agents for ontology analysis and natural language to SQL translation
sqlgraph.sources Flat file loaders: CSV, JSON, XML, XLSX
sqlgraph.core Centralized configuration management

Installation

Prerequisites: Python 3.12+, uv

uv sync

Missing database drivers (psycopg2, pymysql, pyodbc) are detected at runtime and installed automatically.

Configuration

Copy the example configuration file:

cp .env.example .env

Edit .env with your settings. Variables prefixed with SQLGRAPH_ take precedence over plain OPENAI_* variables.

SQLGRAPH_OPENAI_API_KEY=sk-...
SQLGRAPH_OPENAI_BASE_URL=https://api.openai.com/v1
SQLGRAPH_OPENAI_MODEL=gpt-4o

For local models (Ollama, LM Studio, vLLM):

SQLGRAPH_OPENAI_BASE_URL=http://localhost:11434/v1
SQLGRAPH_OPENAI_MODEL=llama3

CLI Usage

From Database

uv run python -m sqlgraph from-db --url sqlite:///mydb.db --name MyOntology --export ontology.json

Without --url, the CLI interactively prompts for connection details and supports SQLite, PostgreSQL, MySQL, or custom SQLAlchemy URLs.

After ontology generation, the CLI asks which database should store the ontology persistently.

From Flat Files

uv run python -m sqlgraph from-file data.csv --name Products
uv run python -m sqlgraph from-file data.json
uv run python -m sqlgraph from-file data.xlsx

Query Ontology with LLM

uv run python -m sqlgraph query ontology.json --question "Which employees joined after 2020?"

API Usage

Generate Ontology from Database

from sqlgraph.db.connector import DatabaseConnector
from sqlgraph.db.discoverer import SchemaDiscoverer
from sqlgraph.ontology.builder import OntologyBuilder
from sqlgraph.ontology.exporter import OntologyExporter

connector = DatabaseConnector("sqlite:///demo.db")
discoverer = SchemaDiscoverer(connector)
schema = discoverer.discover()
ontology = OntologyBuilder.from_schema(schema, name="Demo")

OntologyExporter.to_json(ontology, "ontology.json")

Generate Ontology from CSV

from sqlgraph.sources.loader import FileLoader
from sqlgraph.ontology.exporter import OntologyExporter

ontology = FileLoader.load("data.csv")
OntologyExporter.to_markdown(ontology)

Natural Language to SQL

from sqlgraph.agents.ontology_agent import OntologyAgent

agent = OntologyAgent()
sql = agent.query(ontology, "Which employees joined after 2020?")
print(sql)

Automatic Driver Installation

When connecting to a database, SQLGraph checks whether the required driver is installed. If missing, it installs it automatically via pip:

Dialect Auto-installed Package
PostgreSQL psycopg2-binary
MySQL / MariaDB pymysql
MS SQL Server pyodbc
Oracle cx_oracle

Ontology Export Formats

  • JSON: Structured representation for programmatic consumption
  • SQL DDL: CREATE TABLE statements reflecting the ontology structure
  • Markdown: Human-readable tables for documentation

Project Structure

sqlgraph/
├── core/
│   └── config.py          # Centralized configuration
├── db/
│   ├── connector.py         # SQLAlchemy engine + auto driver install
│   └── discoverer.py      # Schema inspection and FK resolution
├── ontology/
│   ├── models.py            # Entity, Relationship, Attribute dataclasses
│   ├── builder.py           # Ontology construction from schema or DataFrame
│   └── exporter.py          # JSON, DDL, Markdown export
├── llm/
│   └── client.py            # OpenAI-compatible chat client
├── agents/
│   └── ontology_agent.py    # Agno agent for analysis and SQL generation
├── sources/
│   └── loader.py            # CSV, JSON, XML, XLSX ingestion
├── cli.py                   # Typer CLI
└── __main__.py              # Entry point: python -m sqlgraph

License

See License file for details.