Skip to content

test(db): provision the test database the way production does - #254

Merged
HugoFara merged 1 commit into
mainfrom
fix/test-db-provisioning
Jul 25, 2026
Merged

test(db): provision the test database the way production does#254
HugoFara merged 1 commit into
mainfrom
fix/test-db-provisioning

Conversation

@HugoFara

Copy link
Copy Markdown
Owner

The test database was missing three tables and carried the wrong type for languages.LgID, so whole areas of the suite exercised nothing while appearing to pass.

Correction first

I originally reported this as a production bug — that 20251223_173100_add_local_dictionaries.sql fails on any FK-enforcing install. That was wrong. 20251221_120000_add_inter_table_foreign_keys.sql:13 widens languages.LgID to int(11), and it runs before the dictionary migration (Dec 21 < Dec 23). Migrations::checkAndUpdate() applies baseline.sql and then runs every pending migration in order, so production is fine. The defect is confined to test provisioning.

What was wrong

tests/setup_test_db.php applied baseline.sql and then marked every migration as applied without running it, on the premise that "the baseline already includes all table structures from migrations".

That premise is false for anything added after the baseline was last regenerated. books, local_dictionaries and local_dictionary_entries exist only in migrations, so those tables never existed in the test database while their migrations were recorded as applied. Every test touching them skipped silently — 35 of the 44 in DictionaryFacadeTest, including the regression tests added for #250.

Skipping 20251221_120000_add_inter_table_foreign_keys.sql also left languages.LgID at the baseline's tinyint(3). Later migrations declare FK columns as int(11) to match production, and creating such a table against a tinyint parent fails with errno 150 — which FOREIGN_KEY_CHECKS=0 does not suppress. So even had the migrations run, local_dictionaries would not have been created.

The change

Migrations now actually run, in order, statement by statement, skipping any already recorded and tolerating per-statement failures exactly as Migrations::update() does (legacy migrations reference tables the modern baseline no longer has). Three further adjustments were needed to land on a schema that matches production:

  • Finish the column widening under modern table names. The FK migration targets textitems2 and newsfeeds; the baseline creates those as word_occurrences and news_feeds, so its statements for them no-op and left the columns too narrow for their foreign keys.
  • Add FK constraints unconditionally, skipping ones already present, rather than gating the whole block on whether the FK migration was recorded. That gate meant a database whose migrations had run got no FK constraints at all.
  • Clear orphaned rows before adding a constraint. The main suite drops every foreign key (Migrations::dropAllForeignKeys, reached via the restore and migration paths) and can leave children whose parent is gone. This script also runs non-destructively before an integration run, so without the cleanup the constraint cannot be re-added — which surfaced as ForeignKeyTest::testTagDeleteCascadesToWordTags failing once the test actually started running.

Result

before after
tables 16 20
languages.LgID tinyint(3) int(11) (as production)
FK constraints 14 28
integration skips 25 13
integration assertions 356 384

Twelve more integration tests actually run. The 9085-test suite is unchanged (9085 passed, same assertion count). Verified passing in both orderings: on a freshly reset database, and on one left dirty by a full suite run — the latter being the case that previously failed. test:setup-db is idempotent again (a second invocation adds nothing; it previously drifted 20 → 27 tables). Psalm clean; PHPCS unchanged from main's baseline for this file.

The test database was missing three tables and carried the wrong type
for languages.LgID, so whole areas of the suite exercised nothing.

setup_test_db.php applied baseline.sql and then marked every migration
as applied without running it, on the premise that "the baseline already
includes all table structures from migrations". That premise is false
for anything added after the baseline was last regenerated: `books`,
`local_dictionaries` and `local_dictionary_entries` appear only in
migrations, so those tables never existed here while their migrations
were recorded as applied. Tests touching them skipped in silence,
locally and on CI -- 35 of the 44 in DictionaryFacadeTest, including the
regression tests for (#250).

Skipping 20251221_120000_add_inter_table_foreign_keys.sql also left
languages.LgID at the baseline's tinyint(3). Production widens it to
int(11) there, and later migrations declare FK columns as int(11) to
match. Creating such a table against a tinyint parent fails with errno
150, which FOREIGN_KEY_CHECKS=0 does not suppress -- so even had the
migrations run, local_dictionaries would not have been created.

Migrations now actually run, in order, statement by statement, skipping
any already recorded and tolerating per-statement failures exactly as
Migrations::update() does (legacy migrations reference tables the modern
baseline no longer has). Three further adjustments were needed to get a
schema that matches production:

- Finish the column widening under modern table names. The FK migration
  targets `textitems2` and `newsfeeds`, which the baseline creates as
  `word_occurrences` and `news_feeds`, so its statements for them no-op
  and left those columns too narrow for their foreign keys.
- Add the FK constraints unconditionally, skipping ones already present,
  rather than gating the whole block on whether the FK migration was
  recorded. That gate meant a database whose migrations had run got no
  FK constraints at all.
- Clear orphaned rows before adding a constraint. The main suite drops
  every foreign key (Migrations::dropAllForeignKeys, via the restore and
  migration paths) and can leave children whose parent is gone; this
  script also runs non-destructively before an integration run, so
  without the cleanup the constraint cannot be re-added.

A fresh database goes from 16 tables to 20, with languages.LgID at
int(11) and 28 foreign keys instead of 14. The integration suite runs 12
more tests than before (skips 25 -> 13, assertions 356 -> 384), passing
both on a fresh database and on one left dirty by a full suite run. The
9085-test suite is unchanged.

Note that production was never affected: Migrations::checkAndUpdate
applies baseline.sql and then runs every pending migration in order, so
LgID is already int(11) by the time the dictionary migration runs.
@HugoFara
HugoFara merged commit 8840f90 into main Jul 25, 2026
14 checks passed
@HugoFara
HugoFara deleted the fix/test-db-provisioning branch July 25, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant