Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions eng/common/pipelines/templates/jobs/benchmark-job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
parameters:
- name: BenchmarkProject
type: string
default: 'tools/azsdk-cli/Azure.Sdk.Tools.Cli.Benchmarks'
- name: ScenarioName
type: string
default: ' '
- name: Tags
type: string
default: ' '
- name: Model
type: string
default: ' '
- name: Parallelism
type: number
default: 5

jobs:
- job: Run_Benchmark
variables:
- template: /eng/pipelines/templates/variables/globals.yml
- template: /eng/pipelines/templates/variables/image.yml
- group: 'AzSDK_Eval_Variable_group'
displayName: 'Run Benchmark'
pool:
name: $(LINUXPOOL)
image: $(LINUXVMIMAGE)
os: linux

steps:
- checkout: none

- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
parameters:
SkipCheckoutNone: true
Repositories:
- Name: $(Build.Repository.Name)
Commitish: $(Build.SourceVersion)
WorkingDirectory: $(System.DefaultWorkingDirectory)/$(Build.Repository.Name)
Paths:
- '${{ parameters.BenchmarkProject }}/**'
- 'eng/common/**'

- template: /eng/pipelines/templates/steps/install-dotnet.yml
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job references /eng/pipelines/templates/steps/install-dotnet.yml, but that template file doesn’t exist in this repo (there’s no install-dotnet.yml under eng/pipelines/templates/steps/). If this job is ever invoked, the pipeline will fail at template-expansion time. Either add/sync the missing step template into this repo, or replace this include with the explicit UseDotNet tasks (as done in ai-eval-job.yml) so the job is self-contained.

Suggested change
- template: /eng/pipelines/templates/steps/install-dotnet.yml
- task: UseDotNet@2
displayName: 'Install .NET SDK'
inputs:
packageType: 'sdk'
useGlobalJson: true

Copilot uses AI. Check for mistakes.

- task: PowerShell@2
displayName: 'Run benchmarks'
condition: succeeded()
inputs:
targetType: 'inline'
workingDirectory: '$(System.DefaultWorkingDirectory)/$(Build.Repository.Name)/${{ parameters.BenchmarkProject }}'
pwsh: true
script: |
$cliArgs = @("run")

# Determine scenario selection: name, tags, or all
if ("${{ parameters.ScenarioName }}".Trim()) {
$cliArgs += "${{ parameters.ScenarioName }}".Trim()
} elseif ("${{ parameters.Tags }}".Trim()) {
$cliArgs += "--tags", "${{ parameters.Tags }}".Trim()
} else {
$cliArgs += "--all"
}

# Optional model override
if ("${{ parameters.Model }}".Trim()) {
$cliArgs += "--model", "${{ parameters.Model }}".Trim()
}

$cliArgs += "--parallel", "${{ parameters.Parallelism }}"
$cliArgs += "--cleanup", "always"

Write-Host "Running: dotnet run -- $($cliArgs -join ' ')"
dotnet run -- @cliArgs
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dotnet run -- @cliArgs relies on PowerShell splatting for a native executable; this is version-dependent and can end up passing the literal string @cliArgs instead of expanding the argument array. Use native-arg array expansion instead (e.g., pass $cliArgs as the argument list / invoke via & dotnet ... $cliArgs) so each element is forwarded reliably across pwsh versions.

Suggested change
dotnet run -- @cliArgs
dotnet run -- $cliArgs

Copilot uses AI. Check for mistakes.
env:
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1
DOTNET_CLI_TELEMETRY_OPTOUT: 1
DOTNET_MULTILEVEL_LOOKUP: 0
COPILOT_GITHUB_TOKEN: $(azuresdk-copilot-github-pat)
Loading