Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
6 changes: 6 additions & 0 deletions .changes/unreleased/Added-20260813-120000.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Added
body: 'MySQL support: the new `sql_driver` values `asyncmy` and `pymysql` generate code for [asyncmy](https://pypi.org/project/asyncmy/) (asyncio) and [PyMySQL](https://pypi.org/project/PyMySQL/) (synchronous), using `engine: "mysql"` and the same `?` placeholders as the SQLite drivers - the plugin rewrites them to the drivers'' pyformat `%s` at generation time. Inline `ENUM` (and `SET`) columns generate `enums.py` classes, `tinyint(1)` maps to `bool`, `time` to `datetime.timedelta`, and returned `json` columns stay `str` like on every other driver. `:execlastid` returns `cursor.lastrowid` (`None` when nothing was inserted); `:copyfrom` stays PostgreSQL-only. Reused parameters (`sqlc.arg` or `sqlc.slice` used at several sites) merge into one function argument even though MySQL binds every occurrence separately.'
time: 2026-08-13T12:00:00.0000000Z
custom:
Author: Rayakame
PR: "248"
6 changes: 6 additions & 0 deletions .changes/unreleased/Fixed-20260813-120000.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Fixed
body: 'With `omit_unused_models: true`, an enum referenced only through an overridden column no longer breaks generation in either direction: an overridden enum parameter kept its `enums.X(...)` conversion while the class was filtered away (a `NameError` at import), and an enum used only by overridden return columns was retained as a dead `enums.py` nothing imports.'
time: 2026-08-13T12:00:00.0000000Z
custom:
Author: Rayakame
PR: "248"
6 changes: 6 additions & 0 deletions .changes/unreleased/Fixed-20260813-120001.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Fixed
body: 'Query modules whose only queries return whole model rows (no `:many`) no longer import the row''s column-type modules (`datetime`, `decimal`, ...) into their `TYPE_CHECKING` block: nothing in such a module spells those types, and ruff flagged the imports as unused on generated output.'
time: 2026-08-13T12:00:01.0000000Z
custom:
Author: Rayakame
PR: "248"
63 changes: 62 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,18 @@ jobs:
--health-interval=10s
--health-timeout=5s
--health-retries=5
mysql:
image: mysql:9
Comment thread
rayakame marked this conversation as resolved.
env:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: testdb
ports:
- 3306/tcp # host-port is picked automatically
options: >-
--health-cmd="mysqladmin ping -h 127.0.0.1 -uroot -proot"
--health-interval=10s
--health-timeout=5s
--health-retries=10

strategy:
fail-fast: false
Expand All @@ -54,6 +66,7 @@ jobs:
env:
# GitHub tells you which host-port was assigned via the `job.services` context
POSTGRES_URI: postgres://postgres:postgres@localhost:${{ job.services.postgres.ports['5432'] }}/testdb
MYSQL_URI: mysql://root:root@localhost:${{ job.services.mysql.ports['3306'] }}/testdb
run: |
uv run nox -s pytest -- --coverage

Expand Down Expand Up @@ -310,6 +323,54 @@ jobs:
run: |
uv run nox -s turso_async_check

pymysql:
runs-on: ubuntu-latest
name: "Run pymysql check via nox"
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@ae62891fec2bb8e7d6c99fc78c9fec3a63790f8d # v10.0.0
with:
version: "0.12.3"
python-version: "3.13"

- name: Install sqlc
uses: sqlc-dev/setup-sqlc@bac53b7fb28c039a6c7f5736fd1e89744021bdd6 # v5
with:
sqlc-version: '1.31.1'

- name: Run sqlc verify via nox
run: |
uv run nox -s pymysql_check

asyncmy:
runs-on: ubuntu-latest
name: "Run asyncmy check via nox"
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@ae62891fec2bb8e7d6c99fc78c9fec3a63790f8d # v10.0.0
with:
version: "0.12.3"
python-version: "3.13"

- name: Install sqlc
uses: sqlc-dev/setup-sqlc@bac53b7fb28c039a6c7f5736fd1e89744021bdd6 # v5
with:
sqlc-version: '1.31.1'

- name: Run sqlc verify via nox
run: |
uv run nox -s asyncmy_check

go-test:
runs-on: ubuntu-latest
name: "Run go tests"
Expand Down Expand Up @@ -386,7 +447,7 @@ jobs:
retention-days: 30

ci-done:
needs: [test, upload-coverage, asyncpg, psycopg-async, psycopg-sync, aiosqlite, sqlite3, turso-sync, turso-async, pyright, ruff, go-test, go-lint, test-build]
needs: [test, upload-coverage, asyncpg, psycopg-async, psycopg-sync, aiosqlite, sqlite3, turso-sync, turso-async, pymysql, asyncmy, pyright, ruff, go-test, go-lint, test-build]
if: always() && !cancelled()

runs-on: ubuntu-latest
Expand Down
1 change: 1 addition & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ linters:
- '.*/internal/model\.QueryValue$'
- '.*/internal/model\.Column$'
- '.*/internal/render\.importSpec$'
- '.*/internal/driver\.placeholderStyle$'
- '.*/plugin\.Identifier$'
allow-empty: true
tagliatelle:
Expand Down
44 changes: 28 additions & 16 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,11 @@ A sqlc WASM plugin written in Go that generates Python database code (models +
query functions + enums) from SQL. The plugin is compiled to `wasip1/wasm` and
executed by `sqlc generate`. Supported Python drivers: `asyncpg`,
`psycopg_async`, `psycopg_sync` (PostgreSQL engine); `aiosqlite`, `sqlite3`,
`turso_async`, `turso_sync` (SQLite engine; turso is pyturso, experimental).
`turso_async`, `turso_sync` (SQLite engine; turso is pyturso, experimental);
`pymysql`, `asyncmy` (MySQL engine).
Model types: `dataclass`, `attrs`, `msgspec`, `pydantic`. Command support
differs by family: `:copyfrom` is postgres-only, `:execlastid` is
sqlite/turso-only, `:batch*` is unsupported everywhere.
sqlite/turso/mysql-only, `:batch*` is unsupported everywhere.

Generated code targets Python 3.12+ (PEP 695 type aliases and generics,
`enum.StrEnum`). Generated output must be deterministic and byte-identical
Expand Down Expand Up @@ -82,7 +83,7 @@ Python tooling is uv + nox. One-time setup: `uv sync --group dev`. Requires
uv run nox # all default sessions
uv run nox -s asyncpg # regenerate test/driver_asyncpg via sqlc, then pyright + ruff on it
# (one session per driver: asyncpg, psycopg_async, psycopg_sync,
# aiosqlite, sqlite3, turso_sync, turso_async)
# aiosqlite, sqlite3, turso_sync, turso_async, pymysql, asyncmy)
uv run nox -s asyncpg_check # `sqlc diff` variant: verifies committed generated code is up to date
# (CI uses these; every driver has a *_check session)
uv run nox -s pyright ruff # type-check / lint the test suite itself
Expand All @@ -92,12 +93,14 @@ uv run nox -s pytest # runtime tests (needs postgres, see below)
Extra pytest args pass through after `--`, e.g.
`uv run nox -s pytest -- test/driver_asyncpg/msgspec/test_msgspec_classes.py -k test_name`.

pytest needs a local PostgreSQL, configured via the `POSTGRES_URI` env var
(default `postgresql://root:187187@localhost:5432/root`). CONTRIBUTING.md has
a `docker run` one-liner for it.
pytest needs a local PostgreSQL (`POSTGRES_URI`, default
`postgresql://root:187187@localhost:5432/root`) AND a local MySQL
(`MYSQL_URI`, default `mysql://root:187187@localhost:3306/root`); the
session-end cleanup connects to both unconditionally. CONTRIBUTING.md has
`docker run` one-liners for both.

The full verification loop after a generator change: `go build ./...` ->
rebuild wasm -> `uv run nox` -> the seven `*_check` sessions -> commit the
rebuild wasm -> `uv run nox` -> the nine `*_check` sessions -> commit the
regenerated fixtures together with the Go change (when told to commit).

### Changelog
Expand All @@ -117,25 +120,30 @@ generation pipeline lives in `internal/handler.go`:
referenced by overrides; they resolve before override parsing (the override
inherits the converter's py_type).
2. **`internal/types`** - engine-specific SQL-type -> Python-type mapping
(`postgresql.go`, `sqlite.go`), selected by `GetTypeConversionFunc(engine)`.
(`postgresql.go`, `sqlite.go`, `mysql.go`), selected by
`GetTypeConversionFunc(engine)`.
3. **`internal/transform`** - turns the sqlc catalog/queries into the IR:
`BuildEnums()`, `BuildTables()`, `BuildQueries(tables)`,
`FilterUnusedModels()`. `type.go` builds `PyType` and normalizes
`SQLType` (lowercased once here; every downstream consumer relies on it).
`psycopg_sql.go` rewrites `$N` placeholders to psycopg's `%(pN)s` at IR
build time (a small PostgreSQL lexer: skips strings, dollar quotes, quoted
identifiers, nested comments; doubles literal `%`). `plainParams`
pre-reserves every local the driver bodies emit (`conn`/`self`, `sql` for
slice queries, psycopg's `sql_params`/`cur`/`row`/`_decode_hook`); a new
local in a driver body needs a matching seed or a param can shadow it.
`psycopg_sql.go` and `mysql_sql.go` rewrite placeholders at IR build time
(psycopg: `$N` -> `%(pN)s`; MySQL: `?` -> `%s` with `%` doubled - small
SQL lexers matching each engine's rules). `plainParams` pre-reserves
every local the driver bodies emit (`conn`/`self`, `sql` for slice
queries, psycopg's and MySQL's `sql_params`/`cur`/`row`/`_decode_hook`);
a new local in a driver body needs a matching seed or a param can shadow
it. It also merges MySQL's per-occurrence duplicates of reused NAMED
parameters (`IsNamedParam`; the `Repeated` flag keeps their binding
slots) - bare `?` params that share a column name stay distinct.
4. **`internal/model`** - the IR structs (`Enum`, `Table`, `Query`, `PyType`,
...) plus naming logic: initialisms, table-name singularization
(jinzhu/inflection; exclusions match bare AND schema-qualified names),
Python reserved-word escaping (`reserved.go`), and `DedupName`.
5. **`internal/driver`** - the `Driver` interface (`driver.go`) with four
5. **`internal/driver`** - the `Driver` interface (`driver.go`) with five
implementations: `asyncpg.go`; `psycopg.go` for BOTH psycopg flavors
(parameterized by an async flag); `sqlite_base.go` for BOTH sqlite drivers
(module name + async flag); `turso.go` for BOTH turso flavors. A driver
(module name + async flag); `turso.go` for BOTH turso flavors;
`mysql_base.go` for BOTH MySQL drivers (module name + async flag). A driver
knows which query commands it supports and emits query function bodies and
the `QueryResults` class. `conversion.go` holds the asyncpg conversion set
and the ordered sqlite adapter/converter spec table; adapters are
Expand Down Expand Up @@ -180,6 +188,10 @@ buffer is emitted as an extra output file.
emits only what that module needs (params -> adapters, non-overridden
returns -> converters). psycopg's loader registration follows the same
policy (returned json/jsonb types only).
- The MySQL drivers interpolate pyformat placeholders client-side: rewritten
SQL constants carry `%s`/`%%`, and the slice-expansion/placeholder-scanner
machinery in `internal/driver/common.go` must lex them exactly like the
rewriter emitted them.
- `speedups: true` swaps date/datetime decoding to `ciso8601` (sqlite
converter bodies, turso inline decodes); the import resolver tracks which
variant is emitted.
Expand Down
25 changes: 19 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ that you should follow to ensure that your contribution is at its best.
The easiest way to get exactly that version without a Go toolchain is
[sqlc-bin](https://pypi.org/project/sqlc-bin/), whose package version tracks the
sqlc version: `uv tool install "sqlc-bin==1.31.1"`.
- **Docker** (or a local PostgreSQL) - only needed for the runtime tests.
- **Docker** (or a local PostgreSQL plus a local MySQL) - only needed for the runtime tests.

One-time setup for the Python tooling:

Expand Down Expand Up @@ -67,14 +67,15 @@ sessions run with `uv run nox -s name1 name2`:

| Session | What it does |
|-----------------------------------------------------|----------------------------------------------------------------------------------------|
| `asyncpg`, `psycopg_async`, `psycopg_sync`, `sqlite3`, `aiosqlite`, `turso_sync`, `turso_async` | Regenerate the driver's test fixtures via sqlc, then pyright + ruff |
| `asyncpg`, `psycopg_async`, `psycopg_sync`, `sqlite3`, `aiosqlite`, `pymysql`, `asyncmy`, `turso_sync`, `turso_async` | Regenerate the driver's test fixtures via sqlc, then pyright + ruff |
| the `_check` variants of the driver sessions | `sqlc diff` variant: verify the committed generated code is up to date (CI uses these) |
| `pyright` | Type-check the repository |
| `ruff_check` | Non-mutating format + lint check (the CI gate) |
| `ruff`, `ruff_format` | Format and auto-fix the repository - these sessions rewrite files |
| `pytest` | Runtime tests against real databases |

The `pytest` session needs a local PostgreSQL. The connection URI is read from the
The `pytest` session needs BOTH a local PostgreSQL and a local MySQL - the
session-end cleanup connects to each unconditionally. The connection URI is read from the
Comment thread
rayakame marked this conversation as resolved.
`POSTGRES_URI` environment variable and defaults to
`postgresql://root:187187@localhost:5432/root`; set the variable only if your instance
differs from that. To start a matching instance with docker, run
Expand All @@ -88,11 +89,23 @@ docker run --rm --name sqlc-gen-better-python-postgres \
-d postgres
```

and stop it (after running the tests) with the command below; `--rm` removes the container
on stop, so the `docker run` command above can be reused as is next time.
It also needs a local MySQL for the `pymysql` and `asyncmy` suites, read from
`MYSQL_URI` with the default `mysql://root:187187@localhost:3306/root`:

```bash
docker run --rm --name sqlc-gen-better-python-mysql \
-e MYSQL_ROOT_PASSWORD=187187 \
-e MYSQL_DATABASE=root \
-p 3306:3306 \
-d mysql:9
```

Stop the containers (after running the tests) with the commands below; `--rm` removes a
container on stop, so the `docker run` commands above can be reused as is next time.

```bash
docker stop sqlc-gen-better-python-postgres
docker stop sqlc-gen-better-python-mysql
```

Extra pytest arguments pass through after `--`, e.g.
Expand All @@ -103,7 +116,7 @@ Extra pytest arguments pass through after `--`, e.g.
1. Change the Go code and run `make tests` / `make lint`.
2. Rebuild the WASM plugin (see above).
3. `uv run nox` - regenerates the fixtures and runs every check on them. The default
sessions include `pytest`, so have the PostgreSQL from the section above running.
sessions include `pytest`, so have the PostgreSQL and MySQL from the section above running.
The `_check` sessions are not needed locally: they verify committed fixtures
against a fresh regeneration, which is what CI does with the files you commit.
4. If your change affects generated output, add coverage: a query/schema case in the test matrix
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,10 @@ Questions or feedback? Join the [Discord](https://discord.gg/hikari).

- **Four model types** - `dataclass`, `attrs`, `msgspec`, or `pydantic`
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/model-types/)).
- **Seven drivers** - `asyncpg`, `psycopg_async`, and `psycopg_sync` for
PostgreSQL, `aiosqlite` and `sqlite3` for SQLite, plus experimental
`turso_async` and `turso_sync` for [Turso](https://github.com/tursodatabase/turso)
- **Nine drivers** - `asyncpg`, `psycopg_async`, and `psycopg_sync` for
PostgreSQL, `aiosqlite` and `sqlite3` for SQLite, `asyncmy` and `pymysql`
for MySQL, plus experimental `turso_async` and `turso_sync` for
[Turso](https://github.com/tursodatabase/turso)
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/drivers/)).
- **Typed query functions** - one module per query file, one function per query
([docs](https://sqlc-gen-better-python.rayakame.dev/docs/guide/writing-queries/)).
Expand Down
6 changes: 3 additions & 3 deletions docs/content/_index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: sqlc-gen-better-python
description: >-
A sqlc plugin that generates type-safe Python from SQL: dataclass, attrs, msgspec, or pydantic models plus fully typed query functions for asyncpg, psycopg, sqlite3, aiosqlite, and turso.
A sqlc plugin that generates type-safe Python from SQL: dataclass, attrs, msgspec, or pydantic models plus fully typed query functions for asyncpg, psycopg, sqlite3, aiosqlite, asyncmy, pymysql, and turso.
layout: hextra-home
---

Expand Down Expand Up @@ -36,9 +36,9 @@ layout: hextra-home
subtitle="Generate dataclass, attrs, msgspec, or pydantic models - pick per codegen block."
>}}
{{< hextra/feature-card
title="Seven drivers"
title="Nine drivers"
link="docs/guide/drivers"
subtitle="asyncpg and psycopg for PostgreSQL, aiosqlite and sqlite3 for SQLite, and experimental turso support."
subtitle="asyncpg and psycopg for PostgreSQL, aiosqlite and sqlite3 for SQLite, asyncmy and pymysql for MySQL, and experimental turso support."
Comment thread
rayakame marked this conversation as resolved.
>}}
{{< hextra/feature-card
title="Strictly typed output"
Expand Down
Loading
Loading