Skip to content

Bugs/639734 master disable mx cdfi date boundary flaky tests#9513

Open
v-mjofre wants to merge 11 commits into
mainfrom
bugs/639734-master-Disable-mx-cdfi-date-boundary-flaky-tests
Open

Bugs/639734 master disable mx cdfi date boundary flaky tests#9513
v-mjofre wants to merge 11 commits into
mainfrom
bugs/639734-master-Disable-mx-cdfi-date-boundary-flaky-tests

Conversation

@v-mjofre

@v-mjofre v-mjofre commented Jul 15, 2026

Copy link
Copy Markdown

Error Reported

Several CFDI payment stamp tests in codeunit 144001 (MX CFDI) were intermittently failing in RunALTests_MX_Bucket5 due to date-related assertion mismatches. Failures occurred primarily around UTC date boundaries and were caused by inconsistencies between dates generated during test execution and dates returned after timezone conversions.

Examples of observed failures included:

  • Mismatches between the date portion of Comprobante/@Fecha and the expected stamp date.
  • Mismatches between Comprobante/@Fecha and pago20:Pago/@FechaPago when both were expected to represent the same logical date.
  • Exact date comparisons failing when timezone conversion shifted the resulting date by ±1 day relative to UTC.

Root Cause
The tests were vulnerable to two independent date-related issues:

  1. Multiple calls to Today() during test execution could return different dates when execution crossed a UTC midnight boundary, causing inconsistent values between test setup and assertions.

  2. Exact date equality assertions assumed that generated XML dates would always match the expected date precisely. However, timezone conversion during stamp generation can legitimately shift a date by one day relative to UTC, causing otherwise valid results to fail the tests.

Together, these issues produced flaky and non-deterministic test failures.

Solution
The affected tests were updated to make date validation deterministic and resilient to timezone variations:

  • Captured Today() once at the beginning of each test into a local variable (StampDate) and reused it consistently throughout the test execution.
  • Replaced direct calls to Today() in work date initialization, date calculations, and assertions with the captured value.
  • Introduced AssertDateWithinOneDayTolerance to validate dates with a ±1 day tolerance when timezone conversion may legitimately affect the resulting date.
  • Added ParseISODate helper procedure to convert ISO-formatted XML date values into AL Date values for validation.
  • Renamed PaymentStampDatesMatchWhenPostingDateIsToday to PaymentStampDatesNearStampDateWhenPostingIsToday to reflect the updated validation semantics.

Files Modified
App/Layers/MX/Tests/Local/MXCFDI.Codeunit.al

Updated tests:

  • PaymentStampFechaUsesStampDateWhenPostingDateDiffers
  • PaymentStampDatesNearStampDateWhenPostingIsToday
  • PaymentStampOtherXmlAttributesUnchangedByDateFix

Added helper procedures:

  • AssertDateWithinOneDayTolerance
  • ParseISODate

Result
The CFDI payment stamp tests are now deterministic and significantly more robust against environmental timing and timezone-related variations. Date values are captured consistently throughout test execution, and assertions correctly account for legitimate timezone-induced shifts while continuing to validate the expected business behavior. No new compile errors were introduced.

*Fixes
AB#642132
AB#639734

Bug 642132: Instabilities in RunALTests_MX_Bucket5
Bug 639734: [NAV][master]Disable MX CFDI date-boundary flaky tests

v-rosariom
v-rosariom previously approved these changes Jul 15, 2026
var
ActualDate: Date;
DaysDiff: Integer;
DateAssertionLbl: Label '%1. Expected: %2, Actual: %3 (difference: %4 days, tolerance: +/-1 day for timezone shifts)', Comment = '%1 = Error message, %2 = Expected date, %3 = Actual date, %4 = Days difference', Locked = true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Style} \quad \color{gray}{\texttt{\small Iteration\ 1}}$

DateAssertionLbl is declared inside the local procedure AssertDateWithinOneDayTolerance's var block instead of the codeunit's top-level var block, where every other Label in this codeunit is declared (for example ValueErr at line 33).

Procedure-local Labels are still compiled but are fragile in XLIFF extraction and hidden from object-level review. Move the declaration to the codeunit's top-level var block.

Knowledge:

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Copilot PR Review

Iteration 7 · Outcome: completed

Knowledge source: https://github.com/microsoft/BCQuality@186d8a131465475c79244d994acb872cd5c0d4bf

Findings by domain

Findings split into Knowledge-backed (cite a BCQuality article) and Agent (the agent's own judgement, no matching BCQuality rule).

Domain Findings Knowledge-backed Agent Inline Fallback
Style 1 1 0 0 0
Testing 1 0 1 0 0

Totals: 1 knowledge-backed · 1 agent findings.

Orchestrator pre-filter (2 file(s) excluded)

  • layer-disabled (knowledge) : 2 file(s)

Findings produced by the AL review agent v1.7.3. Reply 👎 on any inline comment to flag false positives.

@github-actions github-actions Bot added this to the Version 29.0 milestone Jul 15, 2026
@v-mjofre

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Microsoft"

exit(DMY2Date(DayInt, MonthInt, YearInt));
end;

local procedure AssertDateWithinOneDayTolerance(ExpectedDate: Date; ActualDateText: Text; ErrorMessage: Text)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Testing} \quad \color{gray}{\texttt{\small Iteration\ 2}}$

The test previously asserted Comprobante/@Fecha and pago20:Pago/@FechaPago equal exact dates; the diff replaces that with a new AssertDateWithinOneDayTolerance helper that accepts up to +/-1 day of drift.

The added comments state this is because 'timezone conversions in the production code use different paths for Fecha vs FechaPago, which can shift dates by +/-1 day depending on the server timezone.' That is a description of a real production inconsistency (two code paths computing the same conceptual date differently under timezone conversion) being tolerated by the test rather than fixed at the source. Widening the assertion hides the underlying defect instead of pinning production behavior to a single, deterministic UTC/local conversion; a CI run on a server in a different timezone can now silently pass even though the CFDI/SAT payment-stamp XML would report an off-by-one-day Fecha or FechaPago, which is compliance-relevant data for Mexican tax stamps. If the impact is confirmed, the underlying conversion inconsistency in production (not just the test) should be fixed so both fields derive their date from the same source, and the test tolerance should be tightened back to an exact match. Note: because this finding has no dedicated knowledge-file citation, it is capped at 'minor' here, but the underlying production risk (regulatory/compliance date data drifting across environments) would otherwise warrant 'major' or higher and should be triaged as such.

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

Copilot AI requested a review from a team July 16, 2026 14:16
{
"codeunitId": 144001,
"codeunitName": "MX CFDI",
"method": "PaymentStampDatesMatchWhenPostingDateIsToday",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Testing} \quad \color{gray}{\texttt{\small Iteration\ 3}}$

Tests-Local.DisabledTest.json renames the tracked method for bug 639636 from "PaymentStampDatesMatchWhenPostingDateIsToday" to "PaymentStampDatesNearStampDateWhenPostingIsToday", but the actual test procedure in MXCFDI.Codeunit.al was not renamed and is still called PaymentStampDatesMatchWhenPostingDateIsToday.

The disabled-test entry no longer matches any real test method, so the test-skip tracking for bug 639636 becomes a dead entry: the test will no longer be recognized as disabled by that mechanism. Either rename the AL procedure to match the new JSON name, or revert the JSON "method" value back to "PaymentStampDatesMatchWhenPostingDateIsToday" so the two stay in sync.

Suggested fix (apply manually — could not be anchored as a one-click suggestion):

    "method": "PaymentStampDatesMatchWhenPostingDateIsToday",

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

NamespaceCFD4Txt: Label 'http://www.sat.gob.mx/cfd/4';
SchemaLocationCFD4Txt: Label 'http://www.sat.gob.mx/sitio_internet/cfd/4/cfdv40.xsd';
CertificateNotExistErr: Label 'The Isolated Certificate does not exist. Identification fields and values: Code=''%1''', Comment = '%1 - Isolated Certificate code';
DateAssertionLbl: Label '%1. Expected: %2, Actual: %3 (difference: %4 days, tolerance: +/-1 day for timezone shifts)', Comment = '%1 = Error message, %2 = Expected date, %3 = Actual date, %4 = Days difference', Locked = true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Style} \quad \color{gray}{\texttt{\small Iteration\ 4}}$

DateAssertionLbl combines Locked = true with four placeholders but keeps the Lbl suffix.

The knowledge article's Best Practice pairs Locked = true with the Tok suffix for short tokens or the Txt suffix for diagnostic/telemetry strings carrying placeholders that must stay untranslated; Lbl is reserved for captions and tooltips. Since this string is a non-translatable assertion-diagnostic message (not a caption), rename it (e.g. DateAssertionTxt) so the suffix matches the actual usage.

Knowledge:

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

CopyStr(FechaValue, 1, 10),
CopyStr(FechaPagoValue, 1, 10),
'Comprobante/@Fecha and FechaPago must have the same date when the posting date equals the stamp request date');
AssertDateWithinOneDayTolerance(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Testing} \quad \color{gray}{\texttt{\small Iteration\ 4}}$

In the '[SCENARIO 497244] ...

contain the same date' test, the direct Assert.AreEqual comparing Comprobante/@Fecha to pago20:Pago/@FechaPago was replaced by two independent AssertDateWithinOneDayTolerance calls, each only checked against StampDate. This drops the scenario's actual guarantee that the two dates equal each other: the test can now pass even when Fecha and FechaPago diverge from each other by up to 2 days (e.g. Fecha = StampDate-1 and FechaPago = StampDate+1 both satisfy the +/-1 day tolerance independently), which is exactly the regression this scenario exists to catch. Keep a direct comparison between the two values in addition to the tolerance checks against StampDate so the 'same date' guarantee is still verified. This would be major impact if backed by a curated rule; capped to minor as an agent finding.

Suggested fix (apply manually — could not be anchored as a one-click suggestion):

        AssertDateWithinOneDayTolerance(
            StampDate, FechaValue,
            'Comprobante/@Fecha must be close to the stamp request date');
        AssertDateWithinOneDayTolerance(
            StampDate, FechaPagoValue,
            'FechaPago must be close to the posting date (which equals stamp request date)');
        Assert.AreEqual(
            CopyStr(FechaValue, 1, 10),
            CopyStr(FechaPagoValue, 1, 10),
            'Comprobante/@Fecha and FechaPago must still have the same date when the posting date equals the stamp request date');

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why


local procedure AssertDateWithinOneDayTolerance(ExpectedDate: Date; ActualDateText: Text; ErrorMessage: Text)
var
ActualDate: Date;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Style} \quad \color{gray}{\texttt{\small Iteration\ 5}}$

In the new local procedure AssertDateWithinOneDayTolerance, the var block declares ActualDate: Date before DaysDiff: Integer.

CodeCop AA0021's canonical primitive ordering places Integer before Date, so the declaration order should be swapped.

Suggested fix (apply manually — could not be anchored as a one-click suggestion):

        DaysDiff: Integer;
        ActualDate: Date;

Knowledge:

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

@JesperSchulz JesperSchulz added the Finance GitHub request for Finance area label Jul 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

$\textbf{🟡\ Medium\ Severity\ —\ Testing} \quad \color{gray}{\texttt{\small Iteration\ 6}}$

In OnRun_PaymentStampDatesMatchWhenPostingEqualsStampDate (around line 7441), the added comment states 'timezone conversions in the production code use different paths for Fecha vs FechaPago, which can shift dates by +/-1 day depending on the server timezone.

We verify both values are close to the expected date rather than comparing them directly.' The code then calls AssertDateWithinOneDayTolerance for FechaValue and FechaPagoValue individually (lines 7450-7455), but immediately afterward (lines 7456-7459) still asserts CopyStr(FechaValue,1,10) exactly equals CopyStr(FechaPagoValue,1,10) via Assert.AreEqual. If the stated timezone-shift rationale is real, Fecha and FechaPago can each independently land +/-1 day away from StampDate on different servers, which means the direct equality between the two values can fail even though both are individually within tolerance - reintroducing exactly the flakiness this PR is fixing elsewhere in the same file. Either drop the direct equality assertion (since both values are already validated against StampDate) or replace it with a tolerance-based comparison consistent with the two calls above it.

Line mapping was unavailable, so this was posted as an issue comment.

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Finance GitHub request for Finance area

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants