Feat #470: Remove Qbeast Provider from the Catalog

[osopardo1]

Description

Solution for Qbeast-io/qbeast-spark-private#4. Instead of using the "qbeast" alias in the provider, we will switch to the underlying table format selected. This PR is a proposal and I prefer to review the conceptual feature before proceeding with the opening of the PR.

Type of change

New feature.

The internal components change include:

• QbeastCatalog and QbeastCatalogUtils. Instead of just checking the provider name (which can still be "qbeast", to preserve backward compatibility), it allows other type of Catalog Table Metadata to be manipulated as Qbeast. The characteristics a table needs to have in order to be read and write with Qbeast Implementation are:
  • The provider is "qbeast"
  • The provider is one of the supported providers listed on QbeastOptions. (For the moment, only Delta is part of the list). AND the configuration of the Table includes Qbeast Related information such as columnsToIndex, cubeSize, columnStats orqbeast.revision...
• QbeastTableImpl. The implementation of Spark's Table has another variable called tableFormat. This represents the underlying Provider in which the Table would be saved and represented to the users.
• QbeastOptions. Now it does check that the tableFormat is to one of the supported formats. Otherwise, it would throw an error.

Checklist:

Here is the list of things you should do before submitting this pull request:

• New feature / bug fix has been committed following the Contribution guide.
• Add logging to the code following the Contribution guide.
• Add comments to the code (make it easier for the community!).
• Change the documentation.
• Add tests.
• Your branch is updated to the main branch (dependent changes have been merged).

How Has This Been Tested? (Optional)

The tests are split within:

• QbeastOptionsTest.
• QbeastDataSourceTest.
• QbeastTableImplTest.

[osopardo1]

Closing it due to inactivity. Would be reopened when we have time to dedicate.