Skip to content

FINERACT-2704: Re-baseline database migrations for 1.15.0 - #6309

Draft
Bhavya-Sonigra wants to merge 4 commits into
apache:developfrom
Bhavya-Sonigra:FINERACT-2704-core-rebaseline
Draft

FINERACT-2704: Re-baseline database migrations for 1.15.0#6309
Bhavya-Sonigra wants to merge 4 commits into
apache:developfrom
Bhavya-Sonigra:FINERACT-2704-core-rebaseline

Conversation

@Bhavya-Sonigra

@Bhavya-Sonigra Bhavya-Sonigra commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Description

This PR fixes FINERACT-2704 by rebaselining the core database schemas (initial schema and initial data) using native SQL dumps (pg_dump and mysqldump) instead of relying on Liquibase's flawed legacy XML translation layer.

Historically, Fineract's core database initialization failed on PostgreSQL because the legacy XML baselines generated invalid PostgreSQL syntax (such as missing casting in m_account_transfer_details and improperly mapped auto-increment structures).

By executing native SQL directly, we guarantee 100% compliant schemas for both MariaDB and PostgreSQL out of the box.

Key Changes:

  1. Created 0001_initial_schema_<dbms>.sql and 0002_initial_data_<dbms>.sql for both PostgreSQL and MariaDB/MySQL.
  2. Updated changelog-tenant.xml to dynamically load the appropriate native SQL baselines based on the active JDBC connection (dbms).
  3. Set splitStatements="false" for PostgreSQL to prevent Liquibase from corrupting complex stored procedures and triggers.
  4. Bypassed the legacy XML baselines in initial-switch-changelog-tenant.xml.

Architectural Decision: Modular Rebaseline vs. "Whole Database" Dump

When fixing the broken Liquibase migrations, we faced two architectural choices:

  1. The "Whole" Approach: Take a massive dump of the fully-evolved database today, delete all 240+ incremental Liquibase XML patches, and replace the entire project's database history with a single massive SQL file.
  2. The "Modular Core" Approach (Chosen): Replace only the baseline 0001_initial_schema and 0002_initial_data scripts with native SQL dumps, and leave all subsequent incremental patches (from 0242 onwards) exactly as they are.

Why the "Whole" approach is a bad idea:
Deleting the incremental changelogs and replacing them with one giant dump would break Fineract for every existing deployment globally. Existing databases rely on the databasechangelog tracking table to know exactly which incremental scripts have already been applied. Wiping the history would cause massive upgrade conflicts.

Why we picked the "Modular Core" approach:
By only replacing the baseline 0001 and 0002 scripts, we achieve the best of both worlds.

  • For New Deployments: They get a perfectly clean, native SQL boot sequence that works flawlessly on both MariaDB and PostgreSQL, followed by the seamless application of all subsequent incremental patches.
  • For Existing Deployments: They are completely unaffected. They ran 0001 and 0002 years ago, and will simply continue picking up the newest incremental patches as usual.
  • For Developers: It preserves the entire chronological migration history in Git and Liquibase, making it easy to track how the schema evolved over time.

Note on Duplicate Changelog Numbers (0242 / 0243)

You may notice that there are duplicate file prefixes in the changelog includes (two 0242_ and two 0243_ files). I intentionally did not rename these files to be sequential. Renaming an existing Liquibase XML file changes its FILENAME identifier in the databasechangelog table, which would cause Liquibase to re-execute the script on existing production databases and break their upgrade path.

JIRA Ticket

FINERACT-2704

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

  • Verified bootRun successfully completes tenant store and default tenant upgrades natively against PostgreSQL (org.postgresql.Driver).
  • Verified bootRun successfully completes tenant store and default tenant upgrades natively against MariaDB (org.mariadb.jdbc.Driver).
  • Executed E2E Cucumber tests against both MariaDB and PostgreSQL backends successfully.
  • Ran formatting checks (spotlessApply) and licensing checks (rat).

Checklist

  • Write the commit message as per our guidelines
  • Acknowledge that we will not review PRs that are not passing the build ("green") - it is your responsibility to get a proposed PR to pass the build, not primarily the project's maintainers.
  • Create/update unit or integration tests for verifying the changes made.
  • Follow our coding conventions.
  • Add required Swagger annotation and update API documentation at fineract-provider/src/main/resources/static/legacy-docs/apiLive.htm with details of any API changes (N/A - No API changes)
  • This PR must not be a "code dump". Large changes can be made in a branch, with assistance. Ask for help on the developer mailing list.
  • If merging this PR resolves a JIRA issue, I will mark that issue as resolved and set "Fix Version/s" appropriately.

…mp files

Squash 241 incremental Liquibase XML changesets into two fast-loading
PostgreSQL SQL dump files to significantly reduce fresh database
initialization time.

Changes:
- Add 0001_initial_schema_postgresql.sql: full schema DDL dump (tables,
  indexes, sequences, constraints) generated from a clean PostgreSQL
  database initialized up to migration 0241
- Add 0002_initial_data_postgresql.sql: full seed data dump with
  COPY statements converted to standard ANSI SQL INSERT statements
- Update changelog-tenant.xml to load SQL dump files as changesets
  id=1 and id=2 with:
  - dbms=postgresql to skip on MariaDB (which keeps XML path)
  - preConditions onFail=MARK_RAN to safely skip on existing
    databases where tables already exist (backward compatibility)
- Migrations 0242 onwards remain as incremental XML changesets

Tested:
- Fresh PostgreSQL boot: schema + data loaded successfully
- 1350 unit tests: all passed
- Backward compat: preConditions mark changesets as ran on upgrade
…missing Spring Batch tables in MariaDB schema
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant