Improve integration and unit tests#1031
Open
Avery-Dunn wants to merge 12 commits into
Open
Conversation
Update test dependencies and build plugins
Improve assertion usage patterns
…emove broken B2C browser tests - Upgrade Selenium 3.14.0 to 4.13.0 (last Java 8-compatible) - Add SeleniumDiagnostics: screenshot, HTML dump, browser log capture on failure - Add SeleniumTestWatcher (AfterTestExecutionCallback) for auto-diagnostics - Add WebDriverProvider interface for test extension access to WebDriver - Add CIAMLoginPage page object for CIAM-specific login flows - Add findWithFallback() helper for resilient element locators - Add fallback locator arrays to AzureADLoginPage and ADFSLoginPage - Add failsafe rerunFailingTestsCount=1 for transient browser flakiness - Enable browser console logging via goog:loggingPrefs - Fix DeviceCodeIT close() to quit() for proper WebDriver cleanup - Route CIAM/OIDC authorities to correct page objects in SeleniumTest - Fix scope bug in AcquireTokenInteractiveIT (string literal vs constant) - Remove broken B2C Selenium tests (B2C coverage via ROPC tests instead) - Remove B2CLocalLoginPage and related dead code/constants Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…eAD/microsoft-authentication-library-for-java into avdunn/test-standardization
Contributor
There was a problem hiding this comment.
Pull request overview
This PR focuses on improving the quality, reliability, and maintainability of MSAL4J’s unit and integration tests. It standardizes test naming/assertion patterns, consolidates repeated integration-test logic into shared helpers, updates test/build dependencies, and significantly refactors Selenium-based integration test infrastructure (including diagnostics capture on failure).
Changes:
- Standardize and clean up unit tests (class naming, assertion argument ordering, remove debug output, reduce duplication).
- Refactor integration tests to use shared helpers and a shared in-memory token cache persistence implementation.
- Migrate Selenium 3→4 and harden Selenium infrastructure (fallback locators, CIAM page object, diagnostics capture, WebDriver lifecycle fixes), plus add a JaCoCo warning-only coverage gate and update Maven plugins/dependencies.
Reviewed changes
Copilot reviewed 45 out of 45 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/WSTrustRequestTest.java | Fix assertion argument ordering in XML escaping test. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/TokenRequestExecutorTest.java | Fix assertion argument ordering in token executor tests. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/TelemetryTest.java | Rename test class, remove debug prints, and standardize assertions. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/ServerTelemetryTest.java | Rename test class to singular form. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/OnBehalfOfTest.java | Rename test class to singular form. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/MsalOauthAuthorizationGrantTest.java | Fix typo in test class name. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/MexParserTest.java | Fix assertion argument ordering in Mex parser tests. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/ManagedIdentityTest.java | Rename test class to singular form. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/JsonCompatibilityTest.java | Rename test class/constructor to singular form. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/HttpHeaderTest.java | Fix assertion argument ordering across header tests. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/HelperAndUtilityTest.java | Rename test class and adjust assertions in helper/utility tests. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/DateTimeTest.java | Rename test class to singular form. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/CacheTest.java | Rename test class to singular form. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/CacheFormatTest.java | Rename test class to singular form. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/AuthorizationRequestUrlParametersTest.java | Fix assertion argument ordering and expected values ordering. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/AuthorityTest.java | Fix assertion argument ordering across authority tests. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/AcquireTokenSilentlyTest.java | Remove duplicate resource reader and use shared TestHelper.readResource. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/AccountTest.java | Fix assertion argument ordering in account/cache tests. |
| msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/AadInstanceDiscoveryTest.java | Fix assertion argument ordering in instance discovery assertions. |
| msal4j-sdk/src/integrationtest/java/infrastructure/WebDriverProvider.java | Add interface to expose WebDriver for JUnit extensions. |
| msal4j-sdk/src/integrationtest/java/infrastructure/SeleniumTestWatcher.java | Add JUnit 5 extension to capture diagnostics on Selenium test failures. |
| msal4j-sdk/src/integrationtest/java/infrastructure/SeleniumExtensions.java | Selenium 4 migration, enable browser logging, add fallback locator helper, route CIAM login. |
| msal4j-sdk/src/integrationtest/java/infrastructure/SeleniumDiagnostics.java | Add diagnostics capture (screenshot, HTML, console logs) with basic sensitive-param redaction. |
| msal4j-sdk/src/integrationtest/java/infrastructure/pageobjects/CIAMLoginPage.java | Add CIAM-specific page object for multi-step CIAM login flow. |
| msal4j-sdk/src/integrationtest/java/infrastructure/pageobjects/B2CLocalLoginPage.java | Remove B2C local login page object (tests removed/strategy changed). |
| msal4j-sdk/src/integrationtest/java/infrastructure/pageobjects/AzureADLoginPage.java | Harden Azure AD page object with fallback locators and Selenium 4 WebDriverWait APIs. |
| msal4j-sdk/src/integrationtest/java/infrastructure/pageobjects/ADFSLoginPage.java | Harden ADFS page object with fallback locators and Selenium 4 WebDriverWait APIs. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/UsernamePasswordIT.java | Refactor to use IntegrationTestHelper helpers; remove stray unused builder call. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/TokenCacheIT.java | Standardize assertions and use shared TestTokenCachePersistence. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/TestTokenCachePersistence.java | Add shared in-memory token cache persistence helper for integration tests. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/TestConstants.java | Remove B2C constants no longer needed after B2C Selenium test removal. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/SeleniumTest.java | Add SeleniumTestWatcher integration and CIAM/OIDC routing changes; remove B2C Selenium login branch. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/RefreshTokenIT.java | Refactor assertions to use IntegrationTestHelper. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/OnBehalfOfIT.java | Refactor repeated assertions into IntegrationTestHelper and remove local helper. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/IntegrationTestHelper.java | Expand shared builders, acquisition helpers, and assertion utilities for integration tests. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/HttpClientIT.java | Use IntegrationTestHelper ROPC/acquisition helpers and shared assertions. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/DeviceCodeIT.java | Re-enable device code Selenium test, add diagnostics watcher, and fix driver teardown (quit vs close). |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/ClientCredentialsIT.java | Refactor repeated assertions into IntegrationTestHelper. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/CachePersistenceIT.java | Use shared TestTokenCachePersistence instead of duplicating persistence logic. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/AzureEnvironmentIT.java | Refactor to shared helpers and assertions. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/AuthorizationCodeIT.java | Remove B2C interactive browser tests and re-enable CIAM auth code test path. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/ApacheHttpClientAdapter.java | Fix method typo (builGetRequest → buildGetRequest). |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/AcquireTokenSilentIT.java | Replace local assertion helpers with IntegrationTestHelper assertions. |
| msal4j-sdk/src/integrationtest/java/com/microsoft/aad/msal4j/AcquireTokenInteractiveIT.java | Remove B2C interactive browser tests, re-enable CIAM test, fix scope constant usage. |
| msal4j-sdk/pom.xml | Update test/build dependencies and plugins, add JaCoCo check goal, Selenium 4 deps, and rerunFailingTestsCount. |
Comments suppressed due to low confidence (1)
msal4j-sdk/src/test/java/com/microsoft/aad/msal4j/HelperAndUtilityTest.java:81
- This assertion still has the assertEquals arguments reversed (expected, actual). Since methodReturnedMap is the value under test (returned from AuthorizationRequestUrlParameters.requestParameters()), it should be the "actual" value, and convertedInternalMap should be the "expected" value derived from the internal state.
Comment on lines
+53
to
55
| assertEquals(XML_ESCAPED_DATA, WSTrustRequest.escapeXMLElementData(DATA_TO_ESCAPE)); | ||
| assertEquals(WSTrustRequest.escapeXMLElementData(DATA_TO_ESCAPE), | ||
| StringEscapeUtils.escapeXml10(DATA_TO_ESCAPE)); |
Comment on lines
+108
to
+112
| content.append(pageSource); | ||
|
|
||
| Path destination = getOutputPath(filePrefix + ".html"); | ||
| Files.write(destination, content.toString().getBytes("UTF-8")); | ||
| LOG.info("Page source saved: {}", destination); |
Comment on lines
+142
to
+146
| } | ||
|
|
||
| Path destination = getOutputPath(filePrefix + "-console.log"); | ||
| Files.write(destination, content.toString().getBytes("UTF-8")); | ||
| LOG.info("Browser logs saved: {}", destination); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR aims to resolve some minor-but-widespread issues in integration and unit tests, update all of our test dependencies and build plugins, re-enable several disabled tests, a refactor the Selenium-based test infrastructure to be much more robust.
All changes generally fall into one of the following categories
None of the changes will impact the released source code in any way, and other than the handful of tests mentioned in 5 that were re-enabled/removed none of the changes will affect code coverage.
1. Code Quality Cleanup
*Tests.javato singular*Test.java(e.g.,CacheTests→CacheTest,TelemetryTests→TelemetryTest) to follow standard JUnit conventionsMsalOauthAuthorizatonGrantTest→MsalOauthAuthorizationGrantTestApacheHttpClientAdapter.builGetRequest()→buildGetRequest()assertEquals(actual, expected)calls across 11 test files to use correctassertEquals(expected, actual)orderingSystem.out.println/printstatements fromTelemetryTestreadResource()method inAcquireTokenSilentlyTest(now uses sharedTestHelper.readResource())2. Code Coverage Gate
checkgoal topom.xmlwithhaltOnFailure=false(warning-only)mvn verify, providing early warning if coverage regresses3. Shared Test Helpers
IntegrationTestHelperfrom ~2 methods to ~12 methods:createPublicApp(),createCca()— standard app buildersacquireTokenByRopc(),acquireTokenSilently()(with overloads) — common token acquisition patternsassertAccessTokenNotNull(),assertAccessAndIdTokensNotNull(),assertAccessTokensEqual()/NotEqual(),assertTokensAreEqual()/NotEqual()— assertion helpersTestTokenCachePersistencefrom duplicate inner classes inCachePersistenceITandTokenCacheIT4. Dependency and Plugin Updates
All updated to the latest versions compatible with a Java 8 build JDK:
OkHttp and Apache HttpClient were previously pulled in transitively by Selenium 3 and needed explicit declarations after upgrading to Selenium 4.
5. Selenium Infrastructure Overhaul
Selenium 3 → 4 migration:
WebDriverWait(driver, long)calls toWebDriverWait(driver, Duration)(7 call sites)--headlessto--headless=newin ChromeOptionsDiagnostic infrastructure (new):
SeleniumDiagnostics— captures screenshots, HTML page source (with sensitive parameter redaction), and browser console logs on test failure, saved totarget/selenium-diagnostics/SeleniumTestWatcher— JUnit 5AfterTestExecutionCallbackthat automatically triggers diagnostic capture when a Selenium test fails (runs before@AfterEachso the WebDriver is still alive)WebDriverProvider— interface for test classes to expose their WebDriver to extensionsgoog:loggingPrefsin ChromeOptions for browser console log captureResilient element locators:
findWithFallback(driver, timeout, By... locators)toSeleniumExtensions— single wait loop that checks multiple locators per poll cycle, logs warnings when fallback locators matchAzureADLoginPage,ADFSLoginPagefor username, password, and action button elementsNew page objects:
CIAMLoginPage— handles CIAM-specific multi-step login flow (username → Next → password → Sign in) with correct locators for*.ciamlogin.compagesSeleniumTest.runSeleniumAutomatedLogin()to route CIAM and CIAM-like OIDC authorities to the new page objectTest reliability:
rerunFailingTestsCount=1to Failsafe plugin config for transient browser flakinessDeviceCodeFlowADTestandacquireTokenWithAuthorizationCode_CiamCud(both now passing)AcquireTokenInteractiveITwhere string literal"TestConstants.USER_READ_SCOPE"was used instead of the constantDeviceCodeITusingclose()instead ofquit()for WebDriver cleanupB2C browser test removal:
UsernamePasswordIT(matching MSAL .NET's B2C test strategy, which also only tests B2C via ROPC)B2CLocalLoginPagepage object,performLocalLogin(), and 7 now-unused B2C constants