Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,34 @@ important operational fixes.
Recent Updates
==============

v0.58.1 - Migration template configuration
------------------------------------------------------------------------------

**Fixed:**

* ``migration_config`` accepts the three keys that customize generated
migration files: ``templates``, ``default_format``, and ``title``. The
key validation added in v0.58.0 did not recognize them, so any configuration
using a customized migration template raised
:class:`~sqlspec.exceptions.ImproperConfigurationError` at construction.
``default_format`` was additionally reported with a suggestion to use
``default_schema``, an unrelated setting.
* Template overrides are validated when the configuration is built rather than
when a migration is generated. A misspelling inside ``templates.sql`` or
``templates.py`` reports its full path and the closest valid key, and an
override that is not a mapping is named along with the type supplied.
* :class:`~sqlspec.config.MigrationConfig` documents the real default for
``version_table_name``, which is ``ddl_migrations``.

**Added:**

* :class:`~sqlspec.config.MigrationTemplates`,
:class:`~sqlspec.config.SQLTemplateOverride`, and
:class:`~sqlspec.config.PythonTemplateOverride` describe the template
override shape, so type checkers now cover it.
* The migrations guide documents template customization, including the
placeholders available to each fragment.

v0.58.0 - Configuration and storage correctness
------------------------------------------------------------------------------

Expand Down
12 changes: 12 additions & 0 deletions docs/reference/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,18 @@ Extension Configuration Types
:members:
:show-inheritance:

.. autoclass:: MigrationTemplates
:members:
:show-inheritance:

.. autoclass:: SQLTemplateOverride
:members:
:show-inheritance:

.. autoclass:: PythonTemplateOverride
:members:
:show-inheritance:

.. autoclass:: EventsConfig
:members:
:show-inheritance:
Expand Down
61 changes: 60 additions & 1 deletion docs/usage/migrations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ Common keys
* - ``script_location``
- Migrations directory. Defaults to ``migrations``.
* - ``version_table_name``
- Tracking table name. Defaults to ``sqlspec_migrations``.
- Tracking table name. Defaults to ``ddl_migrations``.
* - ``enabled``
- Set ``False`` to exclude this configuration from CLI operations.
* - ``strict_ordering``
Expand Down Expand Up @@ -305,6 +305,65 @@ configuration is built does not re-run discovery.
it compiles its own modules, the migration sources must remain on disk, since
Python migrations are read and compiled at runtime.

Migration File Templates
------------------------

``create-migration`` renders new files from a built-in template. Three
``migration_config`` keys adjust what it writes:

.. list-table::
:header-rows: 1
:widths: 24 76

* - Key
- Purpose
* - ``default_format``
- Format used when the command is run without ``--file-type``. Either
``sql`` or ``py``. Defaults to ``sql``.
* - ``title``
- Title rendered into generated files. Defaults to ``SQLSpec Migration``.
* - ``templates``
- Fragment overrides for the ``sql`` and ``py`` templates.

Overrides replace individual fragments; anything omitted keeps its default:

.. code-block:: python

config = DuckDBConfig(
connection_config={"database": "/tmp/analytics.db"},
migration_config={
"title": "Acme Migration",
"default_format": "py",
"templates": {
"sql": {
"header": "-- {title} [{adapter}]",
"metadata": ["-- Version: {version}", "-- Owner: {author}"],
}
},
},
)

Every fragment is rendered with ``str.format``, so these placeholders are
available: ``title``, ``version``, ``message``, ``description``, ``created_at``,
``author``, ``adapter``, ``project_slug``, and ``slug`` (the filename-safe form
of the message). An unknown placeholder raises
:class:`~sqlspec.migrations.templates.TemplateValidationError` when the file is
generated.

The SQL template accepts ``header``, ``metadata``, ``body``, and
``description_key``; the Python template accepts ``docstring``, ``imports``,
``body``, and ``description_key``. ``description_key`` names the label the
description is read back from, and takes a string or a list of strings.

.. note::

A body override owns the whole migration body, including the
``-- name: migrate-{version}-up`` and ``-- name: migrate-{version}-down``
markers for SQL, or the ``up``/``down`` functions for Python. SQLSpec does
not merge fragments into a replaced body.

See :class:`~sqlspec.config.MigrationTemplates` for the full override shape.

Output and Logging
------------------

Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ maintainers = [{ name = "Litestar Developers", email = "hello@litestar.dev" }]
name = "sqlspec"
readme = "README.md"
requires-python = ">=3.10, <4.0"
version = "0.58.0"
version = "0.58.1"

[project.urls]
Discord = "https://discord.gg/litestar"
Expand Down Expand Up @@ -331,7 +331,7 @@ opt_level = "3" # Maximum optimization (0-3)
allow_dirty = true
commit = false
commit_args = "--no-verify"
current_version = "0.58.0"
current_version = "0.58.1"
ignore_missing_files = false
ignore_missing_version = false
message = "chore(release): bump to v{new_version}"
Expand Down
138 changes: 126 additions & 12 deletions sqlspec/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@

__all__ = (
"MIGRATION_CONFIG_KEYS",
"MIGRATION_TEMPLATES_KEYS",
"ADKConfig",
"AsyncConfigT",
"AsyncDatabaseConfig",
Expand All @@ -62,11 +63,14 @@
"LifecycleConfig",
"LitestarConfig",
"MigrationConfig",
"MigrationTemplates",
"NoPoolAsyncConfig",
"NoPoolSyncConfig",
"OpenTelemetryConfig",
"PoolT",
"PrometheusConfig",
"PythonTemplateOverride",
"SQLTemplateOverride",
"SanicConfig",
"StarletteConfig",
"SyncConfigT",
Expand Down Expand Up @@ -114,6 +118,55 @@ class LifecycleConfig(TypedDict):
on_error: NotRequired[list[Callable[[Exception, str, dict[str, Any]], None]]]


class SQLTemplateOverride(TypedDict):
"""Overrides for the SQL migration template."""

header: NotRequired[str]
"""First line of the generated file. Supports the migration template placeholders."""

metadata: NotRequired["list[str]"]
"""Comment lines rendered beneath the header."""

body: NotRequired[str]
"""Migration body containing the up and down named statements."""

description_key: NotRequired["str | list[str]"]
"""Metadata label(s) the description is read back from. Defaults to 'Description'."""


class PythonTemplateOverride(TypedDict):
"""Overrides for the Python migration template."""

docstring: NotRequired[str]
"""Module docstring contents. Supports the migration template placeholders."""

body: NotRequired[str]
"""Module body defining the up and down functions."""

imports: NotRequired["list[str]"]
"""Import lines rendered between the docstring and the body."""

description_key: NotRequired["str | list[str]"]
"""Docstring label(s) the description is read back from. Defaults to 'Description'."""


class MigrationTemplates(TypedDict):
"""Template overrides applied when generating migration files.

Placeholders available to every fragment: ``title``, ``version``, ``message``,
``description``, ``created_at``, ``author``, ``adapter``, and ``project_slug``.
"""

sql: NotRequired[SQLTemplateOverride]
"""Overrides for generated ``.sql`` migrations."""

py: NotRequired[PythonTemplateOverride]
"""Overrides for generated ``.py`` migrations."""

title: NotRequired[str]
"""Title used when ``MigrationConfig.title`` is omitted."""


class MigrationConfig(TypedDict):
"""Configuration options for database migrations.

Expand All @@ -124,7 +177,7 @@ class MigrationConfig(TypedDict):
"""Path to the migrations directory. Accepts string or Path object. Defaults to 'migrations'."""

version_table_name: NotRequired[str]
"""Name of the table used to track applied migrations. Defaults to 'sqlspec_migrations'."""
"""Name of the table used to track applied migrations. Defaults to 'ddl_migrations'."""

default_schema: NotRequired[str]
"""Schema applied to migration sessions before user migration SQL runs, when supported by the adapter."""
Expand Down Expand Up @@ -198,30 +251,91 @@ class MigrationConfig(TypedDict):
Defaults to False.
"""

default_format: NotRequired["Literal['sql', 'py']"]
"""File format used by ``create-migration`` when none is requested. Defaults to 'sql'."""

title: NotRequired[str]
"""Title rendered into generated migration files. Defaults to 'SQLSpec Migration'."""

templates: NotRequired[MigrationTemplates]
"""Template fragment overrides applied when generating migration files."""


MIGRATION_CONFIG_KEYS: "frozenset[str]" = MigrationConfig.__required_keys__ | MigrationConfig.__optional_keys__
MIGRATION_TEMPLATES_KEYS: "frozenset[str]" = MigrationTemplates.__required_keys__ | MigrationTemplates.__optional_keys__
_TEMPLATE_FRAGMENT_KEYS: "dict[str, frozenset[str]]" = {
"sql": SQLTemplateOverride.__required_keys__ | SQLTemplateOverride.__optional_keys__,
"py": PythonTemplateOverride.__required_keys__ | PythonTemplateOverride.__optional_keys__,
}


def validate_migration_config_keys(migration_config: "Mapping[str, Any]") -> None:
"""Reject migration configuration keys that SQLSpec does not read.
def _report_unknown_keys(
mapping: "Mapping[str, Any]", valid_keys: "frozenset[str]", prefix: str, scope: str
) -> "list[str]":
"""Describe keys a configuration scope does not declare.

Args:
migration_config: Migration configuration mapping to check.
mapping: Mapping to check.
valid_keys: Keys the scope accepts.
prefix: Dotted path prepended to each reported key.
scope: Scope name used in the valid-key summary, empty for the top level.

Raises:
ImproperConfigurationError: If the mapping contains an unrecognized key.
Returns:
Report lines, empty when every key is recognized.
"""
unknown = sorted(key for key in migration_config if key not in MIGRATION_CONFIG_KEYS)
unknown = sorted(key for key in mapping if key not in valid_keys)
if not unknown:
return
return []

lines = []
for key in unknown:
suggestions = get_close_matches(key, MIGRATION_CONFIG_KEYS, n=1, cutoff=0.6)
suggestions = get_close_matches(key, valid_keys, n=1, cutoff=0.6)
hint = f" Did you mean {suggestions[0]!r}?" if suggestions else ""
lines.append(f"Unknown migration_config key {key!r}.{hint}")
lines.append(f"Valid keys: {', '.join(sorted(MIGRATION_CONFIG_KEYS))}.")
raise ImproperConfigurationError(" ".join(lines))
lines.append(f"Unknown migration_config key {f'{prefix}{key}'!r}.{hint}")
lines.append(f"Valid {scope}keys: {', '.join(sorted(valid_keys))}.")
return lines


def _report_template_keys(templates: Any) -> "list[str]":
"""Describe unrecognized keys nested under ``templates``.

Args:
templates: Value configured for the ``templates`` key.

Returns:
Report lines, empty when the overrides are recognized.
"""
if not isinstance(templates, Mapping):
return [f"migration_config key 'templates' must be a mapping, got {type(templates).__name__}."]

lines = _report_unknown_keys(templates, MIGRATION_TEMPLATES_KEYS, "templates.", "'templates' ")
for section, fragment_keys in _TEMPLATE_FRAGMENT_KEYS.items():
overrides = templates.get(section)
if overrides is None:
continue
path = f"templates.{section}"
if not isinstance(overrides, Mapping):
lines.append(f"migration_config key '{path}' must be a mapping, got {type(overrides).__name__}.")
continue
lines.extend(_report_unknown_keys(overrides, fragment_keys, f"{path}.", f"'{path}' "))
return lines


def validate_migration_config_keys(migration_config: "Mapping[str, Any]") -> None:
"""Reject migration configuration keys that SQLSpec does not read.

Args:
migration_config: Migration configuration mapping to check.

Raises:
ImproperConfigurationError: If the mapping contains an unrecognized key.
"""
lines = _report_unknown_keys(migration_config, MIGRATION_CONFIG_KEYS, "", "")
templates = migration_config.get("templates")
if templates is not None:
lines.extend(_report_template_keys(templates))
if lines:
raise ImproperConfigurationError(" ".join(lines))


class FlaskConfig(TypedDict):
Expand Down
Loading
Loading