Skip to content

Implement pip dependency caching for Azure Pipelines #5267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
unnatikdm wants to merge 7 commits into
base: develop
Choose a base branch
from
+27 −1
Open

Implement pip dependency caching for Azure Pipelines #5267

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
602982b
Implement pip dependency caching for Azure Pipelines
unnatikdm Mar 1, 2026
7d6b466
Fix Windows cache path quoting
unnatikdm Mar 1, 2026
8f28b3c
Remove accidentally added compiled/binary files
unnatikdm Mar 2, 2026
9487352
Add name to AUTHORS and update CHANGELOG
unnatikdm Mar 2, 2026
7beb40f
Add CACHE_VERSION and update pip cache key
unnatikdm Mar 8, 2026
e43e795
Trigger second run for cache hit verification
unnatikdm Mar 8, 2026
4746135
Merge branch 'baseline-test' into add-pip-caching
unnatikdm Mar 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
trigger:
# start a new build for every push
# start a new build for every push (triggering run 2 for cache verification)
batch: False
branches:
include:
Expand All @@ -19,6 +19,7 @@ jobs:
condition: and(succeeded(), ne(variables['Build.SourceBranch'], 'refs/heads/develop')) # skip for PR merges
variables:
MPLBACKEND: agg
CACHE_VERSION: '1'
strategy:
matrix:
Win-Python311-64bit-full:
Expand Down Expand Up @@ -63,6 +64,29 @@ jobs:
versionSpec: $(PYTHON_VERSION)
addToPath: true
architecture: $(PYTHON_ARCH)

- powershell: |
$cachePath = Join-Path $env:LOCALAPPDATA "pip\Cache"
if (!(Test-Path $cachePath)) { New-Item -ItemType Directory -Force -Path $cachePath }
Write-Host "##vso[task.setvariable variable=PIP_CACHE_DIR]$cachePath"
displayName: 'Set pip cache directory (Windows)'
condition: eq(variables['Agent.OS'], 'Windows_NT')

- bash: |
CACHE_DIR="$HOME/.cache/pip"
mkdir -p "$CACHE_DIR"
echo "##vso[task.setvariable variable=PIP_CACHE_DIR]$CACHE_DIR"
displayName: 'Set pip cache directory (Linux)'
condition: eq(variables['Agent.OS'], 'Linux')

- task: Cache@2
inputs:
key: 'pip | "$(Agent.OS)" | "$(PYTHON_VERSION)" | "$(BUILD_TYPE)" | "$(imageName)" | "$(CACHE_VERSION)" | package/requirements.txt'
restoreKeys: |
pip | "$(Agent.OS)" | "$(PYTHON_VERSION)" | "$(BUILD_TYPE)" | "$(imageName)" | "$(CACHE_VERSION)"
pip | "$(Agent.OS)" | "$(PYTHON_VERSION)" | "$(BUILD_TYPE)" | "$(imageName)"
path: $(PIP_CACHE_DIR)
displayName: 'Cache pip'
# a PEP518 compliant wheel build shoud be
# able to build MDAnalysis in an isolated
# environment *before* any deps are installed
Expand Down
1 change: 1 addition & 0 deletions package/AUTHORS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,7 @@ Chronological list of authors
- Harshit Gajjela
- Kunj Sinha
- Ayush Agarwal
- Unnati Kadam

External code
-------------
Expand Down
1 change: 1 addition & 0 deletions package/CHANGELOG
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Enhancements
* Adds support for parsing `.tpr` files produced by GROMACS 2026.0
* Enables parallelization for analysis.diffusionmap.DistanceMatrix
(Issue #4679, PR #4745)
* Implement pip dependency caching for Azure Pipelines (PR #5267)

Changes
* The msd.py inside analysis is changed, and ProgressBar is implemented inside
Expand Down
Loading