Skip to content

Commit a7bf6da

Browse files
authored
Merge pull request #229 from azwanzuharimi/feature/aws_athena
Feat: Added AWS Athena data loader
2 parents c376020 + 0fed94c commit a7bf6da

3 files changed

Lines changed: 570 additions & 2 deletions

File tree

py-src/data_formulator/data_loader/README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,15 @@ The UI automatically provide the query completion option to help user generate q
2222

2323
### Example Implementations
2424

25+
- `AthenaDataLoader`: AWS Athena integration (SQL queries on S3 data lakes)
26+
- `BigQueryDataLoader`: Google BigQuery integration
2527
- `KustoDataLoader`: Azure Data Explorer (Kusto) integration
2628
- `MySQLDataLoader`: MySQL database integration
29+
- `PostgreSQLDataLoader`: PostgreSQL database integration
30+
- `MSSQLDataLoader`: Microsoft SQL Server integration
31+
- `S3DataLoader`: Amazon S3 file integration (CSV, Parquet, JSON)
32+
- `AzureBlobDataLoader`: Azure Blob Storage integration
33+
- `MongoDBDataLoader`: MongoDB integration
2734

2835
### Testing
2936

py-src/data_formulator/data_loader/__init__.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
from data_formulator.data_loader.postgresql_data_loader import PostgreSQLDataLoader
88
from data_formulator.data_loader.mongodb_data_loader import MongoDBDataLoader
99
from data_formulator.data_loader.bigquery_data_loader import BigQueryDataLoader
10+
from data_formulator.data_loader.athena_data_loader import AthenaDataLoader
1011

1112
DATA_LOADERS = {
1213
"mysql": MySQLDataLoader,
@@ -16,7 +17,8 @@
1617
"azure_blob": AzureBlobDataLoader,
1718
"postgresql": PostgreSQLDataLoader,
1819
"mongodb": MongoDBDataLoader,
19-
"bigquery": BigQueryDataLoader
20+
"bigquery": BigQueryDataLoader,
21+
"athena": AthenaDataLoader
2022
}
2123

22-
__all__ = ["ExternalDataLoader", "MySQLDataLoader", "MSSQLDataLoader", "KustoDataLoader", "S3DataLoader", "AzureBlobDataLoader","PostgreSQLDataLoader", "MongoDBDataLoader", "BigQueryDataLoader", "DATA_LOADERS"]
24+
__all__ = ["ExternalDataLoader", "MySQLDataLoader", "MSSQLDataLoader", "KustoDataLoader", "S3DataLoader", "AzureBlobDataLoader", "PostgreSQLDataLoader", "MongoDBDataLoader", "BigQueryDataLoader", "AthenaDataLoader", "DATA_LOADERS"]

0 commit comments

Comments
 (0)