Skip to content

fix: structured config with create_if_missing for non-existent database #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mikkeldamsgaard merged 1 commit into from
Mar 14, 2026
Merged

fix: structured config with create_if_missing for non-existent database #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- `database.default_database` field for structured config: specifies which database to connect to during `create_if_missing` bootstrap. Defaults to `postgres` for PostgreSQL; MySQL connects without selecting a database. Useful when the user does not have access to the default `postgres` database.

### Fixed

- Structured database config now works with `create_if_missing: true` when the target database does not yet exist. Previously, the initial connection included the non-existent database name, causing an immediate connection error. Now initium connects to a bootstrap database first, creates the target, then reconnects. Fixes [#50](https://github.com/KitStream/initium/issues/50).

### Chores

- Added integration tests for structured database connectivity: special-character passwords (URL-reserved chars like `@`, `:`, `/`, `?`, `#`, `%`), PostgreSQL `options` field (`connect_timeout`), and `create_if_missing` with non-existent database ([#50](https://github.com/KitStream/initium/issues/50)).

## [2.0.1] - 2026-03-14

### Fixed
Expand Down
62 changes: 32 additions & 30 deletions docs/seeding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ database:
name: mydb # Database name
options: # Optional. Driver-specific parameters
sslmode: disable
default_database: postgres # Optional. Bootstrap database for create_if_missing
# --- Common ---
tracking_table: initium_seed # Default: "initium_seed"

Expand Down Expand Up @@ -111,36 +112,37 @@ phases:

### Field reference

| Field | Type | Required | Description |
| ----------------------------------------------- | ----------------- | -------- | ---------------------------------------------------------------------- |
| `database.driver` | string | Yes | Database driver: `postgres`, `mysql`, or `sqlite` |
| `database.url` | string | No | Direct database connection URL (cannot combine with structured fields) |
| `database.url_env` | string | No | Environment variable containing the database URL |
| `database.host` | string | No | Database host (structured config; cannot combine with url/url_env) |
| `database.port` | integer | No | Database port (default: 5432 for postgres, 3306 for mysql) |
| `database.user` | string | No | Database user (structured config) |
| `database.password` | string | No | Database password — special characters work without encoding |
| `database.name` | string | No | Database name (structured config) |
| `database.options` | map[string]string | No | Driver-specific connection parameters (e.g. `sslmode: disable`) |
| `database.tracking_table` | string | No | Name of the seed tracking table (default: `initium_seed`) |
| `phases[].name` | string | Yes | Unique phase name |
| `phases[].order` | integer | No | Execution order (lower first, default: 0) |
| `phases[].database` | string | No | Target database name (for create/switch) |
| `phases[].schema` | string | No | Target schema name (for create/switch) |
| `phases[].create_if_missing` | boolean | No | Create the database/schema if it does not exist (default: false) |
| `phases[].timeout` | string | No | Default wait timeout (e.g. `30s`, `1m`, `1m30s`; default: `30s`) |
| `phases[].wait_for[].type` | string | Yes | Object type: `table`, `view`, `schema`, or `database` |
| `phases[].wait_for[].name` | string | Yes | Object name to wait for |
| `phases[].wait_for[].timeout` | string | No | Per-object timeout override (e.g. `60s`, `2m`, `1m30s`) |
| `phases[].seed_sets[].name` | string | Yes | Unique name for the seed set (used in tracking) |
| `phases[].seed_sets[].order` | integer | No | Execution order (lower values first, default: 0) |
| `phases[].seed_sets[].mode` | string | No | Seed mode: `once` (default) or `reconcile` |
| `phases[].seed_sets[].tables[].table` | string | Yes | Target database table name |
| `phases[].seed_sets[].tables[].order` | integer | No | Execution order within the seed set (default: 0) |
| `phases[].seed_sets[].tables[].unique_key` | string[] | No | Columns for duplicate detection |
| `phases[].seed_sets[].tables[].auto_id.column` | string | No | Auto-generated ID column name |
| `phases[].seed_sets[].tables[].auto_id.id_type` | string | No | ID type (default: `integer`) |
| `phases[].seed_sets[].tables[].rows[]._ref` | string | No | Internal reference name for cross-table references |
| Field | Type | Required | Description |
| ----------------------------------------------- | ----------------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
| `database.driver` | string | Yes | Database driver: `postgres`, `mysql`, or `sqlite` |
| `database.url` | string | No | Direct database connection URL (cannot combine with structured fields) |
| `database.url_env` | string | No | Environment variable containing the database URL |
| `database.host` | string | No | Database host (structured config; cannot combine with url/url_env) |
| `database.port` | integer | No | Database port (default: 5432 for postgres, 3306 for mysql) |
| `database.user` | string | No | Database user (structured config) |
| `database.password` | string | No | Database password — special characters work without encoding |
| `database.name` | string | No | Database name (structured config) |
| `database.options` | map[string]string | No | Driver-specific connection parameters (e.g. `sslmode: disable`) |
| `database.default_database` | string | No | Database to connect to during `create_if_missing` bootstrap. Default: `postgres` for PostgreSQL, none for MySQL. |
| `database.tracking_table` | string | No | Name of the seed tracking table (default: `initium_seed`) |
| `phases[].name` | string | Yes | Unique phase name |
| `phases[].order` | integer | No | Execution order (lower first, default: 0) |
| `phases[].database` | string | No | Target database name (for create/switch) |
| `phases[].schema` | string | No | Target schema name (for create/switch) |
| `phases[].create_if_missing` | boolean | No | Create the database/schema if it does not exist (default: false) |
| `phases[].timeout` | string | No | Default wait timeout (e.g. `30s`, `1m`, `1m30s`; default: `30s`) |
| `phases[].wait_for[].type` | string | Yes | Object type: `table`, `view`, `schema`, or `database` |
| `phases[].wait_for[].name` | string | Yes | Object name to wait for |
| `phases[].wait_for[].timeout` | string | No | Per-object timeout override (e.g. `60s`, `2m`, `1m30s`) |
| `phases[].seed_sets[].name` | string | Yes | Unique name for the seed set (used in tracking) |
| `phases[].seed_sets[].order` | integer | No | Execution order (lower values first, default: 0) |
| `phases[].seed_sets[].mode` | string | No | Seed mode: `once` (default) or `reconcile` |
| `phases[].seed_sets[].tables[].table` | string | Yes | Target database table name |
| `phases[].seed_sets[].tables[].order` | integer | No | Execution order within the seed set (default: 0) |
| `phases[].seed_sets[].tables[].unique_key` | string[] | No | Columns for duplicate detection |
| `phases[].seed_sets[].tables[].auto_id.column` | string | No | Auto-generated ID column name |
| `phases[].seed_sets[].tables[].auto_id.id_type` | string | No | ID type (default: `integer`) |
| `phases[].seed_sets[].tables[].rows[]._ref` | string | No | Internal reference name for cross-table references |

### Wait-for object support by driver

Expand Down
55 changes: 54 additions & 1 deletion src/seed/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@ pub mod schema;

use crate::logging::Logger;

fn bootstrap_database(config: &schema::DatabaseConfig) -> String {
if !config.default_database.is_empty() {
return config.default_database.clone();
}
match config.driver.as_str() {
// PostgreSQL requires connecting to an existing database; `postgres` is
// guaranteed to exist on every cluster.
"postgres" | "postgresql" => "postgres".into(),
// MySQL can connect without selecting a database, which avoids needing
// access to the `mysql` system schema.
_ => String::new(),
}
}

fn render_template(content: &str) -> Result<String, String> {
let env_map: std::collections::HashMap<String, String> = std::env::vars().collect();
let mut jinja_env = minijinja::Environment::new();
Expand Down Expand Up @@ -41,9 +55,48 @@ pub fn run(
let tracking_table = plan.database.tracking_table.clone();
let driver = plan.database.driver.clone();

// When using structured config and a phase needs to create a database that
// matches the configured name, we try the normal connection first. If it
// fails, we fall back to connecting to a bootstrap database, create the
// target, then reconnect. See https://github.com/KitStream/initium/issues/50
let may_need_bootstrap = plan.database.has_structured_config()
&& plan.phases.iter().any(|p| {
p.create_if_missing && !p.database.is_empty() && p.database == plan.database.name
});

log.info("connecting to database", &[("driver", driver.as_str())]);

let db = db::connect(&plan.database)?;
let db = match db::connect(&plan.database) {
Ok(db) => db,
Err(err) if may_need_bootstrap => {
log.info(
"target database not reachable, bootstrapping via default database",
&[("driver", driver.as_str())],
);

let mut admin_config = plan.database.clone();
admin_config.name = bootstrap_database(&plan.database);

let mut admin_db = db::connect(&admin_config)?;

for phase in &plan.phases {
if phase.create_if_missing && !phase.database.is_empty() {
log.info(
"creating database if missing",
&[("database", phase.database.as_str())],
);
admin_db.create_database(&phase.database)?;
}
// Schemas are database-scoped, so they must be created after
// reconnecting to the target database. The executor handles
// schema creation in execute_phase().
}
drop(admin_db);

db::connect(&plan.database).map_err(|_| err)?
}
Err(err) => return Err(err),
};
let mut exec = executor::SeedExecutor::new(log, db, tracking_table, reset)
.with_dry_run(dry_run)
.with_reconcile_all(reconcile_all);
Expand Down
43 changes: 43 additions & 0 deletions src/seed/schema.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,8 @@ pub struct DatabaseConfig {
#[serde(default)]
pub name: String,
#[serde(default)]
pub default_database: String,
#[serde(default)]
pub options: HashMap<String, String>,
#[serde(default = "default_tracking_table")]
pub tracking_table: String,
Expand Down Expand Up @@ -529,6 +531,47 @@ phases:
assert_eq!(plan.database.port, None);
}

#[test]
fn test_structured_config_default_database() {
let yaml = r#"
database:
driver: postgres
host: pg.example.com
user: app
name: mydb
default_database: maintenance_db
phases:
- name: phase1
seed_sets:
- name: x
tables:
- table: t
rows: []
"#;
let plan = SeedPlan::from_yaml(yaml).unwrap();
assert_eq!(plan.database.default_database, "maintenance_db");
}

#[test]
fn test_structured_config_default_database_empty_by_default() {
let yaml = r#"
database:
driver: postgres
host: localhost
user: app
name: mydb
phases:
- name: phase1
seed_sets:
- name: x
tables:
- table: t
rows: []
"#;
let plan = SeedPlan::from_yaml(yaml).unwrap();
assert!(plan.database.default_database.is_empty());
}

#[test]
fn test_rejects_url_and_structured_config() {
let yaml = r#"
Expand Down
Loading
Loading