Skip to content

Arm backend: Refactor Arm tester to inherit from test harness#18101

Open
per wants to merge 1 commit intopytorch:mainfrom
per:move_arm_tester_to_test_harness
Open

Arm backend: Refactor Arm tester to inherit from test harness#18101
per wants to merge 1 commit intopytorch:mainfrom
per:move_arm_tester_to_test_harness

Conversation

@per
Copy link
Collaborator

@per per commented Mar 11, 2026

Summary

Move from inheriting from the XNNPack tester to the common test harness instead.

Test plan

Tested through all CI tests.

cc @digantdesai @freddan80 @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell

Move from inheriting from the XNNPack tester to the common
test harness.

Signed-off-by: Per Åstrand <per.astrand@arm.com>
Change-Id: I68840d7a4c402083e6d93ac8a2bb5feadf057dca
@per per requested a review from digantdesai as a code owner March 11, 2026 12:40
Copilot AI review requested due to automatic review settings March 11, 2026 12:40
@pytorch-bot
Copy link

pytorch-bot bot commented Mar 11, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18101

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Awaiting Approval, 1 New Failure, 1 Cancelled Job, 1 Pending

As of commit 9d65477 with merge base 0112ada (image):

AWAITING APPROVAL - The following workflow needs approval before CI can run:

NEW FAILURE - The following job has failed:

CANCELLED JOB - The following job was cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 11, 2026
@per per added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk release notes: arm Changes to the ARM backend delegate labels Mar 11, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the Arm backend test infrastructure to use the common backend test harness (instead of inheriting from the XNNPack tester), aligning Arm’s stage implementations with the shared backends/test/harness pipeline.

Changes:

  • Switched Arm tester stages (Partition, ToEdgeTransformAndLower, ToExecutorch, RunPasses, Serialize) to inherit from executorch.backends.test.harness.stages.
  • Updated ArmTester to register Arm-specific stage classes with the common harness Tester.
  • Updated Arm tests and BUCK deps to reference the harness tester/stages instead of XNNPack’s tester.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
backends/arm/test/tester/serialize.py Migrate Arm Serialize stage to inherit from common harness Serialize.
backends/arm/test/tester/arm_tester.py Core refactor: ArmTester now builds on common harness Tester and stage implementations.
backends/arm/test/ops/test_where.py Update test to use ArmQuantize instead of XNNPack Quantize stage.
backends/arm/test/misc/test_bn_relu_folding_qat.py Switch Quantize import to the common harness tester.
backends/arm/test/BUCK Update dependencies from XNNPack tester target to harness tester target.
Comments suppressed due to low confidence (2)

backends/arm/test/tester/arm_tester.py:336

  • The PR description still contains the template placeholders (e.g. "[PLEASE REMOVE]") and an empty test plan. Please clean these up so the description reflects the actual intent and testing for this change.
class ArmTester(tester.Tester):
    def __init__(
        self,
        model: torch.nn.Module,
        example_inputs: Tuple[Any, ...],

backends/arm/test/tester/arm_tester.py:489

  • The serialize() method exposes a timeout parameter, but the constructed Serialize stage is initialized with timeout=self.timeout instead of the method argument, so per-call overrides are ignored. Use the timeout argument (or remove it from the signature if it’s intentionally not supported).
    def serialize(
        self,
        serialize_stage: Optional[BaseStages.Serialize] = None,
        # Keep timeout keyword-only so positional usage matches the base class.
        *,
        timeout: int = 480,
    ):
        if serialize_stage is None:
            serialize_stage = Serialize(
                compile_spec=self.compile_spec,
                module=self.original_module,
                use_portable_ops=self.use_portable_ops,
                timeout=self.timeout,
            )

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines 169 to 174
constant_methods: Optional[Dict[str, Any]] = None,
transform_passes: Optional[
Union[Sequence[PassType], Dict[str, Sequence[PassType]]]
] = None,
compile_spec: Optional[ArmCompileSpec] = None,
):
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ToEdgeTransformAndLower.__init__ takes a compile_spec parameter but never reads it, and the caller passes compile_spec=self.compile_spec. Either remove this parameter (and stop passing it) or use it to configure the stage so the argument isn’t misleading/dead.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: arm Changes to the ARM backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants