Skip to content

Update service memory mitigation: remove stale system health checks #266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alexbass112 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update service memory mitigation: remove stale system health checks #266

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 31 additions & 14 deletions TSG/Update/Update-Service-terminated-repeatedly-by-ALM.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,10 @@ $installedUpdates = Get-SolutionUpdate | where State -eq Installed
$resourceIds = @( $installedUpdates | % { $_.ResourceId.Split('/')[-1] } )
Write-Host "Found $($installedUpdates.Count) installed updates: $($resourceIds -join ", ")"
$removedHC = $false
if ($resourceIds.Count -gt 0)
{
$removedHC = $false
foreach ($resourceId in $resourceIds)
{
$healthCheckPath = Join-Path "C:\ClusterStorage\Infrastructure_1\Shares\SU1_Infrastructure_1\Updates\HealthCheck" $resourceId
Expand All @@ -78,21 +79,37 @@ if ($resourceIds.Count -gt 0)
Write-Host "No health check found for $resourceId at $healthCheckPath"
}
}
}
if ($removedHC)
$systemHealthPath = Join-Path "C:\ClusterStorage\Infrastructure_1\Shares\SU1_Infrastructure_1\Updates\HealthCheck" "System"
if (Test-Path $systemHealthPath)
{
$oldSystemChecks = Get-ChildItem $systemHealthCheckPath | sort LastWriteTime | select -SkipLast 10
if ($oldSystemChecks)
{
Write-Host "Some health checks were removed, so clearing update service cache."
$clientCert = ls Cert:\LocalMachine\My\ | ? Subject -match "URP" | sort NotAfter | select -last 1
$updateEndpoint = "https://$(Get-ClusterGroup *update* | % OwnerNode | % Name).$($env:USERDNSDOMAIN):4900"
$clearCacheResult = Invoke-WebRequest -Certificate $clientCert -UseBasicParsing -Uri "$updateEndpoint/caches/Update" -Method "Delete"
if ([int]::TryParse($clearCacheResult.Content, [ref]$null))
{
Write-Host "Removed $($clearCacheResult.Content) updates from the cache."
}
else
{
Write-Warning "Unexpected result from clearing the cache: $($clearCacheResult | Out-String)"
}
Write-Host "Removing $(oldSystemChecks.Count) old system health check results."
$removedHC = $true
$oldSystemChecks | Remove-Item -Force -Verbose
Comment on lines +87 to +92
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the system health cleanup block, the script references $systemHealthCheckPath (undefined) instead of $systemHealthPath, and the log message uses $(oldSystemChecks.Count) without $ (will error in the subexpression). Additionally, Select-Object -SkipLast is not available in Windows PowerShell 5.1 (typical default on nodes), so this line will fail unless running PowerShell 7+. Update the path variable reference, fix the subexpression to use $oldSystemChecks, and use a PowerShell 5.1-compatible approach to keep the newest 10 items (e.g., sort descending and skip 10).

Copilot uses AI. Check for mistakes.
}
else
{
Write-Host "No stale health checks found at $systemHealthPath"
}
}
if ($removedHC)
{
Write-Host "Some health checks were removed, so clearing update service cache."
$clientCert = ls Cert:\LocalMachine\My\ | ? Subject -match "URP" | sort NotAfter | select -last 1
$updateEndpoint = "https://$(Get-ClusterGroup "Azure Stack HCI Update Service Cluster Group" | % OwnerNode | % Name).$($env:USERDNSDOMAIN):4900"
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assignment to $updateEndpoint uses a double-quoted string that contains another double-quoted string inside the $() expression (Get-ClusterGroup "Azure Stack HCI Update Service Cluster Group"), which will break PowerShell parsing. Use single quotes for the cluster group name (or build the URL in multiple steps) so the outer string remains valid.

Suggested change
$updateEndpoint = "https://$(Get-ClusterGroup "Azure Stack HCI Update Service Cluster Group" | % OwnerNode | % Name).$($env:USERDNSDOMAIN):4900"
$updateEndpoint = "https://$(Get-ClusterGroup 'Azure Stack HCI Update Service Cluster Group' | % OwnerNode | % Name).$($env:USERDNSDOMAIN):4900"

Copilot uses AI. Check for mistakes.
$clearCacheResult = Invoke-WebRequest -Certificate $clientCert -UseBasicParsing -Uri "$updateEndpoint/caches/Update" -Method "Delete"
if ([int]::TryParse($clearCacheResult.Content, [ref]$null))
{
Write-Host "Removed $($clearCacheResult.Content) updates from the cache."
}
else
{
Write-Host "Result from clearing the cache: $($clearCacheResult | Out-String)"
}
}
```
Expand Down
Loading