Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,17 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
drawing. Non-ASCII in .sql files has corrupted sqlc's byte-offset parameter
rewriting before.

## Writing style (docs, comments, commit messages)

- Lead with the fact. No preamble, no restating the question.
- One idea per sentence. Short sentences. Cut every word that adds nothing.
- Show an example instead of describing behavior in prose.
- Comments explain only what the code cannot: constraints, whys, gotchas.
Never narrate what the next line does.
- No marketing tone ("powerful", "simply", "clean"). No hedging
("should", "generally") unless the uncertainty is real.
- When in doubt, write half as much.

## What this is

A sqlc WASM plugin written in Go that generates Python database code (models +
Expand Down
111 changes: 60 additions & 51 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,48 +7,67 @@
![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json)
[![CI](https://github.com/rayakame/sqlc-gen-better-python/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/rayakame/sqlc-gen-better-python/actions/workflows/ci.yml)

A WASM plugin for SQLC allowing the generation of Python code.
`sqlc-gen-better-python` is a [sqlc](https://sqlc.dev) plugin that turns your
SQL schema and queries into modern, fully typed Python database code: models,
typed query functions, and enums. You keep writing SQL; the Python stays in
sync with it.

The generated code requires **Python 3.12 or newer** (it uses PEP 695 type
aliases and generics, and `enum.StrEnum`).
You write:

> [!TIP]
> Besides the [official installation methods](https://docs.sqlc.dev/en/latest/overview/install.html),
> the `sqlc` CLI itself is also pip-installable via
> [`sqlc-bin`](https://pypi.org/project/sqlc-bin/), which ships the unmodified
> official binaries - no Go toolchain required: `uv add --dev sqlc-bin` (or
> `pip install sqlc-bin`) puts `sqlc` on your PATH, pinnable like any other
> Python dependency.
```sql
-- name: GetUser :one
SELECT * FROM users WHERE id = $1;
```

and get back:

```python
async def get_user(conn: ConnectionLike, *, id_: int) -> models.User | None:
row = await conn.fetchrow(GET_USER, id_)
if row is None:
return None
return models.User(id_=row[0], name=row[1])
```

No ORM, no hand-written row unpacking. Generated code targets **Python 3.12 or
newer** and passes pyright (strict) and ruff.

## Documentation

**https://sqlc-gen-better-python.rayakame.dev/**

- [Getting Started](https://sqlc-gen-better-python.rayakame.dev/docs/getting-started/) - install the plugin and generate your first models.
- [Guide](https://sqlc-gen-better-python.rayakame.dev/docs/guide/) - configuration, drivers, model types, writing queries, and every feature, each with real generated output.
- [Reference](https://sqlc-gen-better-python.rayakame.dev/docs/reference/) - all configuration options, SQL-to-Python type mappings, and per-driver feature support.
- [Guide](https://sqlc-gen-better-python.rayakame.dev/docs/guide/) - every feature, each with real generated output.
- [Reference](https://sqlc-gen-better-python.rayakame.dev/docs/reference/) - all options, type mappings, and per-driver feature support.

Questions or feedback? Join the [Discord](https://discord.gg/hikari).

## Used by

[<img src="docs/static/images/used-by/nmarkov.png" alt="nMarkov logo" height="72">](https://nmarkov.xyz/)

**[nMarkov](https://nmarkov.xyz/)** - a Discord chatbot that learns from your
server's messages and generates its own.
## Features

Using `sqlc-gen-better-python` in your project? [Open an issue](https://github.com/rayakame/sqlc-gen-better-python/issues)
to get listed here.
- **Four model types** - `dataclass`, `attrs`, `msgspec`, or `pydantic`
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/model-types/)).
- **Seven drivers** - `asyncpg`, `psycopg_async`, and `psycopg_sync` for
PostgreSQL, `aiosqlite` and `sqlite3` for SQLite, plus experimental
`turso_async` and `turso_sync` for [Turso](https://github.com/tursodatabase/turso)
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/drivers/)).
- **Typed query functions** - one module per query file, one function per query
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/writing-queries/)).
- **PostgreSQL enums** as `enum.StrEnum` classes
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/enums/)).
- **Type overrides and converters** - swap a column's Python type, or plug in your
own encode/decode functions
([overrides](https://sqlc-gen-better-python.rayakame.dev/docs/guide/type-overrides/),
[converters](https://sqlc-gen-better-python.rayakame.dev/docs/guide/converters/)).
- **Typed JSON columns** via msgspec structs
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/working-with-json/)).
- **Optional docstrings** in `google`, `numpy`, or `pep257` convention
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/docstrings/)).

> [!NOTE]
> Every Release before `v1.0.0`, including this one is an beta release.
> These versions are primarly released for interested people who want to test this plugin and help make it better.
>
> Everything that is implemented works and is being used in production environments already.
> Since `v0.5.0` this includes full support for PostgreSQL enums and a fourth model type, `pydantic`.
> Feel free to lmk any wanted features and I'm going to do my best on implementing them with the time I have rn.
Every [sqlc macro](https://docs.sqlc.dev/en/latest/reference/macros.html) is
supported. Which query commands are available depends on the driver - see the
[feature support matrix](https://sqlc-gen-better-python.rayakame.dev/docs/reference/feature-support/).

## Example Config
## Example config

```yaml
# filename: sqlc.yaml
Expand All @@ -73,35 +92,25 @@ sql:

```

> [!TIP]
> No `sqlc` yet? Besides the [official installation methods](https://docs.sqlc.dev/en/latest/overview/install.html),
> `uv add --dev sqlc-bin` (or `pip install sqlc-bin`) installs
> [`sqlc-bin`](https://pypi.org/project/sqlc-bin/), the unmodified official
> binaries as a pinnable Python package - no Go toolchain required.

More options at the [`sqlc` config reference](https://docs.sqlc.dev/en/stable/reference/config.html),
and the full plugin option list in the
[configuration reference](https://sqlc-gen-better-python.rayakame.dev/docs/reference/configuration-options/).

## Features
## Used by

- **Four model types** - `dataclass`, `attrs`, `msgspec`, or `pydantic`
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/model-types/)).
- **Seven drivers** - `asyncpg`, `psycopg_async`, and `psycopg_sync` for
PostgreSQL, `aiosqlite` and `sqlite3` for SQLite, plus experimental
`turso_async` and `turso_sync` for [Turso](https://github.com/tursodatabase/turso)
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/drivers/)).
- **Typed query functions** - one module per query file, one function per query
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/writing-queries/)).
- **PostgreSQL enums** as `enum.StrEnum` classes
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/enums/)).
- **Type overrides and converters** - swap a column's Python type, or plug in your
own encode/decode functions
([overrides](https://sqlc-gen-better-python.rayakame.dev/docs/guide/type-overrides/),
[converters](https://sqlc-gen-better-python.rayakame.dev/docs/guide/converters/)).
- **Typed JSON columns** via msgspec structs
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/working-with-json/)).
- **Optional docstrings** in `google`, `numpy`, or `pep257` convention
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/docstrings/)).
- Generated code passes **pyright strict** and **ruff**.
[<img src="docs/static/images/used-by/nmarkov.png" alt="nMarkov logo" height="72">](https://nmarkov.xyz/)

Every [sqlc macro](https://docs.sqlc.dev/en/latest/reference/macros.html) is
supported. Which query commands are available depends on the driver - see the
[feature support matrix](https://sqlc-gen-better-python.rayakame.dev/docs/reference/feature-support/).
**[nMarkov](https://nmarkov.xyz/)** - a Discord chatbot that learns from your
server's messages and generates its own.

Using `sqlc-gen-better-python` in your project? [Open an issue](https://github.com/rayakame/sqlc-gen-better-python/issues)
to get listed here.

## Development

Expand Down
3 changes: 0 additions & 3 deletions docs/content/docs/guide/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,6 @@ You never call this plugin directly. You write a SQL schema and queries, and
per table), one query module per query `.sql` file (typed functions), and an
`enums.py` when your schema has enums.

So everything in this guide is driven by two inputs - your SQL and your
`sqlc.yaml` options - and observed through one output: the generated Python.

{{< callout type="info" >}}
New here? Start with [Getting Started](/docs/getting-started) for install and
a first `sqlc generate`, then come back to this guide.
Expand Down
33 changes: 14 additions & 19 deletions docs/content/docs/guide/drivers.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,10 @@ It must match your `engine`. Seven drivers are supported:
Every generated query function takes the connection as its first argument, so you
open and manage the connection yourself and pass it in.

All PostgreSQL drivers produce the same models and type contract, so choosing
between them is about the driver itself: pick `asyncpg` when raw driver
throughput is the priority, and one of the psycopg drivers to stay in the
psycopg ecosystem (libpq, pipeline mode, PgBouncer friendliness) at comparable
speed - `psycopg_async` for asyncio code, `psycopg_sync` for plain synchronous
code.
All PostgreSQL drivers produce the same models and type contract. Pick
`asyncpg` for raw throughput, or a psycopg driver to stay in the psycopg
ecosystem (libpq, pipeline mode, PgBouncer) - `psycopg_async` for asyncio,
`psycopg_sync` for synchronous code.

## asyncpg (PostgreSQL)

Expand All @@ -51,11 +49,10 @@ asyncio.run(main())
asyncpg supports `:copyfrom` (bulk insert via `copy_records_to_table`).

{{< callout type="info" >}}
asyncpg itself ships without strict type annotations - install
[asyncpg-stubs](https://pypi.org/project/asyncpg-stubs/) so that pyright or
mypy understand generated annotations like
`asyncpg.Connection[asyncpg.Record]`. This only affects type checking and
is never evaluated at runtime.
asyncpg ships without strict type annotations - install
[asyncpg-stubs](https://pypi.org/project/asyncpg-stubs/) so pyright and
mypy understand annotations like `asyncpg.Connection[asyncpg.Record]`.
Type checking only; never evaluated at runtime.
{{< /callout >}}

## psycopg_async (PostgreSQL)
Expand Down Expand Up @@ -102,14 +99,12 @@ with psycopg.connect("postgresql://user:pass@localhost/db") as conn:
user = queries.get_field_naming(conn, id_=1)
```

The synchronous flavor of the psycopg driver (Psycopg 3.2 or newer, like
`psycopg_async`): identical models, placeholders, and type contract, emitted
as plain functions with no `async`/`await`. The connection annotation is
`psycopg.Connection[psycopg.rows.TupleRow]`, and `:many` queries return the
same `QueryResults` helper - call it (`queries.list_x(conn)()`) to fetch every
row at once, or iterate it directly with a plain `for` loop. The json/jsonb
raw-text loader registration works exactly as on `psycopg_async`; the Windows
event-loop caveat does not apply.
Same contract as `psycopg_async`, emitted as plain functions without
`async`/`await`. The connection annotation is
`psycopg.Connection[psycopg.rows.TupleRow]`. `:many` returns the same
`QueryResults` helper: call it (`queries.list_x(conn)()`) to fetch every row,
or iterate it with a plain `for` loop. The json/jsonb loader registration is
identical; the Windows event-loop caveat does not apply.

## aiosqlite (async SQLite)

Expand Down
9 changes: 4 additions & 5 deletions docs/content/docs/guide/enums.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,10 @@ class TestMood(enum.StrEnum):
VALUE__HIDDEN = "_hidden"
```

Notice the member names are uppercased, and values that are not valid Python
identifiers are sanitized: the digit-leading `24h` becomes `VALUE_24H` and the
underscore-leading `_hidden` becomes `VALUE__HIDDEN`. The string *values* are
untouched, so round-tripping to the database is exact. See
[Naming and identifiers](/docs/guide/naming) for the full sanitization rules.
Member names are uppercased and invalid identifiers sanitized: `24h` becomes
`VALUE_24H`, `_hidden` becomes `VALUE__HIDDEN`. The string *values* are
untouched, so round-tripping to the database is exact. Full rules in
[Naming and identifiers](/docs/guide/naming).

## Using enums

Expand Down
17 changes: 7 additions & 10 deletions docs/content/docs/guide/sqlite-type-conversion.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,13 +103,10 @@ generated code, so only enable it if you install that package.

## The turso drivers

The registration model on this page does not apply to the experimental
`turso_sync`/`turso_async` drivers: pyturso has no `detect_types` or
adapter/converter registry, so the generated code converts the same declared
types inline instead - no `PARSE_DECLTYPES`, no registration, identical
resulting Python types.

`speedups` works the same way as on the SQLite drivers: it swaps the inline
`date` and `datetime`/`timestamp` decodes for `ciso8601` calls (`decimal`,
`bool`, and `blob` are unchanged) and adds `ciso8601` as a runtime dependency
of the generated code.
None of this applies to the experimental `turso_sync`/`turso_async` drivers:
pyturso has no `detect_types` and no adapter/converter registry. The generated
code converts the same declared types inline - no flags, no registration,
identical Python types.

`speedups` works the same: the inline `date` and `datetime`/`timestamp`
decodes become `ciso8601` calls.
2 changes: 0 additions & 2 deletions docs/content/docs/guide/working-with-json.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@ what a converter's `from_db`/`to_db` pair provides.

## The wire type contract

This is the one rule to internalize:

{{< callout type="info" >}}
A `jsonb` (or `json`) column's wire type is **`str`**. So `to_db` must *return*
`str`, and `from_db` *receives* `str`.
Expand Down
28 changes: 12 additions & 16 deletions docs/content/docs/guide/writing-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,12 @@ async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFiel

### `:one`

Returns the row or `None`. The row is a `models.*` class when the query's columns
match a table, a generated `<Name>Row` class when they do not, or a bare scalar
when the query selects a single column. Shown above.
Returns the row or `None` - a `models.*` class when the columns match a table,
a `<Name>Row` class when they do not, or a bare scalar for single-column
selects. Shown above.

When a query's columns do not match one table exactly (a join, a partial select,
an aggregate), the plugin generates a dedicated `<Name>Row` class instead of
reusing a table model:
A join, partial select, or aggregate gets a dedicated `<Name>Row` class
instead of a table model:

```python
class GetJoinedFieldNamingsRow(msgspec.Struct):
Expand All @@ -73,9 +72,8 @@ async def get_joined_field_namings(conn: ConnectionLike, *, id_: int) -> GetJoin

### `:many`

Returns a `QueryResults[T]` - a helper that supports both async iteration and
one-shot fetching, so you do not pay for materializing the whole result set
unless you want it:
Returns `QueryResults[T]`, which supports both async iteration and one-shot
fetching - the result set is only materialized if you ask for it:

```python
def get_many_test_timestamp_postgres_type(conn: ConnectionLike, *, id_: int) -> QueryResults[datetime.datetime]:
Expand Down Expand Up @@ -166,14 +164,12 @@ this:

## Grouping into a class

With `emit_classes: true`, the standalone functions of each query file become
methods on a class named after that file - `queries_field_namings.sql` yields
`QueriesFieldNamings` - so you get one class per query module rather than one
class overall.
With `emit_classes: true`, each query file's functions become methods on a
class named after the file: `queries_field_namings.sql` yields
`QueriesFieldNamings`, one class per query module.

The connection is passed once to the constructor and is also exposed as a
read-only `conn` property. The bodies are otherwise unchanged, except that `conn`
becomes `self._conn`:
The connection is passed once to the constructor and exposed as a read-only
`conn` property. The bodies only change `conn` to `self._conn`:

```python
class QueriesFieldNamings:
Expand Down
11 changes: 5 additions & 6 deletions docs/content/docs/reference/feature-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,11 @@ generates.

### Prepared queries

Coming from sqlc's Go workflow you might look for an
[`emit_prepared_queries`](https://docs.sqlc.dev/en/latest/howto/prepared_query.html)
equivalent. There is none, on purpose: every stable driver already prepares
statements automatically, so the generated code gets prepared-query
performance without any extra codegen. What differs per driver is *when* a
query gets prepared and which knob controls it:
Looking for sqlc's
[`emit_prepared_queries`](https://docs.sqlc.dev/en/latest/howto/prepared_query.html)?
There is none, on purpose: every stable driver already prepares statements
automatically. What differs is *when* a query is prepared and which knob
controls it:

- **asyncpg** prepares every query it runs and keeps it in a per-connection
LRU statement cache (100 entries by default). Tune it at connect time:
Expand Down
Loading