FINERACT-2704: Re-baseline database migrations for 1.15.0 - #6309
Draft
Bhavya-Sonigra wants to merge 4 commits into
Draft
FINERACT-2704: Re-baseline database migrations for 1.15.0#6309Bhavya-Sonigra wants to merge 4 commits into
Bhavya-Sonigra wants to merge 4 commits into
Conversation
…mp files
Squash 241 incremental Liquibase XML changesets into two fast-loading
PostgreSQL SQL dump files to significantly reduce fresh database
initialization time.
Changes:
- Add 0001_initial_schema_postgresql.sql: full schema DDL dump (tables,
indexes, sequences, constraints) generated from a clean PostgreSQL
database initialized up to migration 0241
- Add 0002_initial_data_postgresql.sql: full seed data dump with
COPY statements converted to standard ANSI SQL INSERT statements
- Update changelog-tenant.xml to load SQL dump files as changesets
id=1 and id=2 with:
- dbms=postgresql to skip on MariaDB (which keeps XML path)
- preConditions onFail=MARK_RAN to safely skip on existing
databases where tables already exist (backward compatibility)
- Migrations 0242 onwards remain as incremental XML changesets
Tested:
- Fresh PostgreSQL boot: schema + data loaded successfully
- 1350 unit tests: all passed
- Backward compat: preConditions mark changesets as ran on upgrade
…missing Spring Batch tables in MariaDB schema
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes FINERACT-2704 by rebaselining the core database schemas (initial schema and initial data) using native SQL dumps (
pg_dumpandmysqldump) instead of relying on Liquibase's flawed legacy XML translation layer.Historically, Fineract's core database initialization failed on PostgreSQL because the legacy XML baselines generated invalid PostgreSQL syntax (such as missing casting in
m_account_transfer_detailsand improperly mapped auto-increment structures).By executing native SQL directly, we guarantee 100% compliant schemas for both MariaDB and PostgreSQL out of the box.
Key Changes:
0001_initial_schema_<dbms>.sqland0002_initial_data_<dbms>.sqlfor both PostgreSQL and MariaDB/MySQL.changelog-tenant.xmlto dynamically load the appropriate native SQL baselines based on the active JDBC connection (dbms).splitStatements="false"for PostgreSQL to prevent Liquibase from corrupting complex stored procedures and triggers.initial-switch-changelog-tenant.xml.Architectural Decision: Modular Rebaseline vs. "Whole Database" Dump
When fixing the broken Liquibase migrations, we faced two architectural choices:
0001_initial_schemaand0002_initial_datascripts with native SQL dumps, and leave all subsequent incremental patches (from0242onwards) exactly as they are.Why the "Whole" approach is a bad idea:
Deleting the incremental changelogs and replacing them with one giant dump would break Fineract for every existing deployment globally. Existing databases rely on the
databasechangelogtracking table to know exactly which incremental scripts have already been applied. Wiping the history would cause massive upgrade conflicts.Why we picked the "Modular Core" approach:
By only replacing the baseline
0001and0002scripts, we achieve the best of both worlds.0001and0002years ago, and will simply continue picking up the newest incremental patches as usual.Note on Duplicate Changelog Numbers (0242 / 0243)
You may notice that there are duplicate file prefixes in the changelog includes (two
0242_and two0243_files). I intentionally did not rename these files to be sequential. Renaming an existing Liquibase XML file changes itsFILENAMEidentifier in thedatabasechangelogtable, which would cause Liquibase to re-execute the script on existing production databases and break their upgrade path.JIRA Ticket
FINERACT-2704
Type of change
How Has This Been Tested?
bootRunsuccessfully completes tenant store and default tenant upgrades natively against PostgreSQL (org.postgresql.Driver).bootRunsuccessfully completes tenant store and default tenant upgrades natively against MariaDB (org.mariadb.jdbc.Driver).spotlessApply) and licensing checks (rat).Checklist
fineract-provider/src/main/resources/static/legacy-docs/apiLive.htmwith details of any API changes (N/A - No API changes)