Skip to content

Commit 0d56779

Browse files
priyabhatnagar25rahuldash171
authored andcommitted
Removed Issue tag Plugin-1669 in Runtime scenarios and test runner file,
updated correct conn string, removed space in yml, removed verify goal in pom. changed sql connector java version to run in github actions and bqmt secret updated
1 parent ab3c7fc commit 0d56779

6 files changed

Lines changed: 10 additions & 15 deletions

File tree

.github/workflows/e2e.yml

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ jobs:
7979
MYSQL_USERNAME:cdapio-github-builds/MYSQL_USERNAME
8080
MYSQL_PASSWORD:cdapio-github-builds/MYSQL_PASSWORD
8181
MYSQL_PORT:cdapio-github-builds/MYSQL_PORT
82-
CONNECTION_STRING:cdapio-github-builds/CONNECTION_STRING
82+
BQMT_CONNECTION_STRING:cdapio-github-builds/BQMT_CONNECTION_STRING
8383
8484
- name: Run required e2e tests
8585
if: github.event_name != 'workflow_dispatch' && github.event_name != 'push' && steps.filter.outputs.e2e-test == 'false'
@@ -89,7 +89,7 @@ jobs:
8989
MYSQL_USERNAME: ${{ steps.secrets.outputs.MYSQL_USERNAME }}
9090
MYSQL_PASSWORD: ${{ steps.secrets.outputs.MYSQL_PASSWORD }}
9191
MYSQL_PORT: ${{ steps.secrets.outputs.MYSQL_PORT }}
92-
CONNECTION_STRING: ${{ steps.secrets.outputs.CONNECTION_STRING }}
92+
BQMT_CONNECTION_STRING: ${{ steps.secrets.outputs.BQMT_CONNECTION_STRING }}
9393

9494
- name: Run all e2e tests
9595
if: github.event_name == 'workflow_dispatch' || github.event_name == 'push' || steps.filter.outputs.e2e-test == 'true'
@@ -99,22 +99,20 @@ jobs:
9999
MYSQL_USERNAME: ${{ steps.secrets.outputs.MYSQL_USERNAME }}
100100
MYSQL_PASSWORD: ${{ steps.secrets.outputs.MYSQL_PASSWORD }}
101101
MYSQL_PORT: ${{ steps.secrets.outputs.MYSQL_PORT }}
102-
CONNECTION_STRING: ${{ steps.secrets.outputs.CONNECTION_STRING }}
102+
BQMT_CONNECTION_STRING: ${{ steps.secrets.outputs.BQMT_CONNECTION_STRING }}
103103

104104
- name: Upload report
105105
uses: actions/upload-artifact@v3
106106
if: always()
107107
with:
108108
name: Cucumber report - ${{ matrix.tests }}
109109
path: ./plugin/target/cucumber-reports
110-
111110
- name: Upload debug files
112111
uses: actions/upload-artifact@v3
113112
if: always()
114113
with:
115114
name: Debug files - ${{ matrix.tests }}
116115
path: ./**/target/e2e-debug
117-
118116
- name: Upload files to GCS
119117
uses: google-github-actions/upload-cloud-storage@v0
120118
if: always()

pom.xml

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
<?xml version="1.0" encoding="UTF-8"?>
2-
32
<!--
43
~ Copyright © 2016-2019 Cask Data, Inc.
54
~
@@ -1195,7 +1194,6 @@
11951194
<execution>
11961195
<goals>
11971196
<goal>integration-test</goal>
1198-
<goal>verify</goal>
11991197
</goals>
12001198
</execution>
12011199
</executions>
@@ -1246,7 +1244,7 @@
12461244
<dependency>
12471245
<groupId>mysql</groupId>
12481246
<artifactId>mysql-connector-java</artifactId>
1249-
<version>8.0.19</version>
1247+
<version>8.0.25</version>
12501248
<scope>test</scope>
12511249
</dependency>
12521250
<dependency>

src/e2e-test/features/bigquerymultitable/MultipleDatabaseTableToBigQueryMultiTableSink_WithMacro.feature

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
@BQMT_SINK
1616
Feature: BigQueryMultiTable sink -Verification of Multiple Database Tables to BigQueryMultiTable successful data transfer using macros
1717

18-
@MULTIPLEDATABASETABLE_SOURCE_TEST @BQMT_Required @PLUGIN-1669
18+
@MULTIPLEDATABASETABLE_SOURCE_TEST @BQMT_Required
1919
Scenario:Verify data is getting transferred from Multiple Database Tables to BQMT sink with all datatypes using macros
2020
Given Open Datafusion Project to configure pipeline
2121
When Expand Plugin group in the LHS plugins list: "Source"
@@ -71,7 +71,7 @@ Feature: BigQueryMultiTable sink -Verification of Multiple Database Tables to Bi
7171
Then Verify the pipeline status is "Succeeded"
7272
Then Validate the values of records transferred to BQMT sink is equal to the value from source MultiDatabase table
7373

74-
@MULTIPLEDATABASETABLE_SOURCE_TEST @BQMT_Required @PLUGIN-1669
74+
@MULTIPLEDATABASETABLE_SOURCE_TEST @BQMT_Required
7575
Scenario:Verify data is getting transferred from Multiple Database Tables to BQMT sink with split field using macros
7676
Given Open Datafusion Project to configure pipeline
7777
When Expand Plugin group in the LHS plugins list: "Source"

src/e2e-test/features/bigquerymultitable/MultipleDatabaseTableToBiqQueryMultiTableSink.feature

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
@BQMT_SINK
1616
Feature: BigQueryMultiTable sink -Verification of MultipleDatabaseTable to BigQueryMultiTable successful data transfer
1717

18-
@MULTIPLEDATABASETABLE_SOURCE_TEST @BQMT_Required @PLUGIN-1669
18+
@MULTIPLEDATABASETABLE_SOURCE_TEST @BQMT_Required
1919
Scenario:Verify data is getting transferred from Multiple Database Tables to BQMT sink with all datatypes
2020
Given Open Datafusion Project to configure pipeline
2121
When Expand Plugin group in the LHS plugins list: "Source"
@@ -58,7 +58,7 @@ Feature: BigQueryMultiTable sink -Verification of MultipleDatabaseTable to BigQu
5858
Then Verify the pipeline status is "Succeeded"
5959
Then Validate the values of records transferred to BQMT sink is equal to the value from source MultiDatabase table
6060

61-
@MULTIPLEDATABASETABLE_SOURCE_TEST @BQMT_Required @PLUGIN-1669
61+
@MULTIPLEDATABASETABLE_SOURCE_TEST @BQMT_Required
6262
Scenario:Verify data is getting transferred from Multiple Database Tables to BQMT sink with split field
6363
Given Open Datafusion Project to configure pipeline
6464
When Expand Plugin group in the LHS plugins list: "Source"

src/e2e-test/java/io/cdap/plugin/bigquerymultitable/runners/TestRunner.java

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,7 @@
2727
features = {"src/e2e-test/features"},
2828
glue = {"io.cdap.plugin.bigquerymultitable.stepsdesign", "io.cdap.plugin.gcs.stepsdesign", "stepsdesign",
2929
"io.cdap.plugin.common.stepsdesign"},
30-
tags = {"@BQMT_SINK and not @PLUGIN-1669"},
31-
//TODO: Enable test once issue is fixed https://cdap.atlassian.net/browse/PLUGIN-1669
30+
tags = {"@BQMT_SINK"},
3231
monochrome = true,
3332
plugin = {"pretty", "html:target/cucumber-html-report/bigquerymultitable",
3433
"json:target/cucumber-reports/cucumber-bigquerymultitable.json",

src/e2e-test/resources/pluginParameters.properties

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -238,7 +238,7 @@ host=MYSQL_HOST
238238
user=MYSQL_USERNAME
239239
pass=MYSQL_PASSWORD
240240
port=MYSQL_PORT
241-
connectionString=CONNECTION_STRING
241+
connectionString=BQMT_CONNECTION_STRING
242242
mySQLSourceTable=dummy
243243
sourceTable=dummy
244244
tableNameField=tableName

0 commit comments

Comments
 (0)