-
Notifications
You must be signed in to change notification settings - Fork 22
ensure testing of built version after deployment #159
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
849dec9
370dd06
329b18f
94cd723
57580f9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| name: 'Wait for PyPI version' | ||
| description: 'Wait for a specific package version to become available on PyPI or TestPyPI' | ||
| inputs: | ||
| repository: | ||
| description: 'PyPI repository type: "pypi" or "testpypi"' | ||
| required: true | ||
| package: | ||
| description: 'Package name' | ||
| required: true | ||
| version: | ||
| description: 'Package version to wait for' | ||
| required: true | ||
| max_attempts: | ||
| description: 'Maximum number of retry attempts' | ||
| required: false | ||
| default: '30' | ||
| wait_seconds: | ||
| description: 'Seconds to wait between attempts' | ||
| required: false | ||
| default: '10' | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Install requests | ||
| shell: bash | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install requests | ||
|
|
||
| - name: Wait for version to be available | ||
| shell: bash | ||
| env: | ||
| REPOSITORY: ${{ inputs.repository }} | ||
| PACKAGE: ${{ inputs.package }} | ||
| VERSION: ${{ inputs.version }} | ||
| MAX_ATTEMPTS: ${{ inputs.max_attempts }} | ||
| WAIT_SECONDS: ${{ inputs.wait_seconds }} | ||
| run: | | ||
| if [ "$REPOSITORY" = "testpypi" ]; then | ||
| API_URL="https://test.pypi.org/pypi/$PACKAGE/json" | ||
| REPO_NAME="TestPyPI" | ||
| elif [ "$REPOSITORY" = "pypi" ]; then | ||
| API_URL="https://pypi.org/pypi/$PACKAGE/json" | ||
| REPO_NAME="PyPI" | ||
| else | ||
| echo "ERROR: repository must be 'pypi' or 'testpypi', got '$REPOSITORY'" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Export shell variables so Python can access them via os.environ | ||
| export API_URL REPO_NAME | ||
|
|
||
| ATTEMPT=0 | ||
| while [ $ATTEMPT -lt $MAX_ATTEMPTS ]; do | ||
| if python -c " | ||
| import os | ||
| import requests | ||
| import sys | ||
| try: | ||
| api_url = os.environ['API_URL'] | ||
| version = os.environ['VERSION'] | ||
| repo_name = os.environ['REPO_NAME'] | ||
| r = requests.get(api_url, timeout=10) | ||
| r.raise_for_status() | ||
| versions = r.json().get('releases', {}) | ||
| print('Available versions:', list(versions.keys())[-10:]) # Show last 10 versions | ||
| if version in versions: | ||
| print(f'✓ Version {version} is available on {repo_name}') | ||
| sys.exit(0) | ||
| else: | ||
| print(f'✗ Version {version} is NOT available on {repo_name}') | ||
| sys.exit(1) | ||
| except Exception as e: | ||
| print(f'Error checking version: {e}') | ||
| sys.exit(1) | ||
| " 2>/dev/null; then | ||
| echo "Version $VERSION is now available on $REPO_NAME" | ||
| exit 0 | ||
| fi | ||
| ATTEMPT=$((ATTEMPT + 1)) | ||
| echo "Attempt $ATTEMPT/$MAX_ATTEMPTS: Version $VERSION not yet available on $REPO_NAME, waiting $WAIT_SECONDS seconds..." | ||
| sleep $WAIT_SECONDS | ||
| done | ||
| echo "ERROR: Version $VERSION did not become available on $REPO_NAME after $MAX_ATTEMPTS attempts" | ||
| exit 1 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,6 +21,8 @@ jobs: | |
| build: | ||
| name: Build package | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| version: ${{ steps.extract-version.outputs.version }} | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
@@ -43,6 +45,24 @@ jobs: | |
| run: | | ||
| twine check dist/* | ||
|
|
||
| - name: Extract package version | ||
| id: extract-version | ||
| run: | | ||
| WHEEL_FILE=$(ls dist/*.whl) | ||
| # Extract version from wheel filename (format: griddataformats-VERSION-py3-none-any.whl) | ||
| VERSION=$(basename "$WHEEL_FILE" | sed -n 's/griddataformats-\([^-]*\)-.*/\1/p') | ||
| # Fallback: install wheel temporarily and get version | ||
| if [ -z "$VERSION" ]; then | ||
| # This is a bit dirty; running in a virtual environment would be cleaner. | ||
| # (pip install only works because our package's dependencies are easy to install.) | ||
| python -m pip install --upgrade pip | ||
| pip install "$WHEEL_FILE" --quiet | ||
| VERSION=$(python -c "import gridData; print(gridData.__version__)") | ||
| pip uninstall -y griddataformats --quiet | ||
| fi | ||
| echo "version=$VERSION" >> $GITHUB_OUTPUT | ||
| echo "Extracted version: $VERSION" | ||
|
|
||
| - name: Upload dist files | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
|
|
@@ -133,7 +153,7 @@ jobs: | |
| fail-fast: false | ||
| matrix: | ||
| os: [ubuntu-latest, macos-latest] | ||
| needs: deploy-testpypi | ||
| needs: [build, deploy-testpypi] | ||
| if: | | ||
| github.repository == 'MDAnalysis/GridDataFormats' && | ||
| (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')) | ||
|
|
@@ -143,10 +163,20 @@ jobs: | |
| with: | ||
| python-version: "3.14" | ||
|
|
||
| - name: Checkout repository for actions | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Wait for version to be available on TestPyPI | ||
| uses: ./.github/actions/wait-for-pypi-version | ||
| with: | ||
| repository: testpypi | ||
| package: GridDataFormats | ||
| version: ${{ needs.build.outputs.version }} | ||
|
|
||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For simpler version, replace custom action with a
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. now uses
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| - name: Install from TestPyPI | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ GridDataFormats[test] | ||
| pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ "GridDataFormats[test]==${{ needs.build.outputs.version }}" | ||
|
|
||
| - name: Test import | ||
| run: | | ||
|
|
@@ -163,7 +193,7 @@ jobs: | |
| fail-fast: false | ||
| matrix: | ||
| os: [ubuntu-latest, macos-latest] | ||
| needs: deploy-pypi | ||
| needs: [build, deploy-pypi] | ||
| if: | | ||
| github.repository == 'MDAnalysis/GridDataFormats' && | ||
| (github.event_name == 'release' && github.event.action == 'published') | ||
|
|
@@ -173,10 +203,20 @@ jobs: | |
| with: | ||
| python-version: "3.14" | ||
|
|
||
| - name: Checkout repository for actions | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Wait for version to be available on PyPI | ||
| uses: ./.github/actions/wait-for-pypi-version | ||
| with: | ||
| repository: pypi | ||
| package: GridDataFormats | ||
| version: ${{ needs.build.outputs.version }} | ||
|
|
||
| - name: Install from PyPI | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install GridDataFormats[test] | ||
| pip install "GridDataFormats[test]==${{ needs.build.outputs.version }}" | ||
|
|
||
| - name: Test import | ||
| run: | | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a brief initial look - it feels like most of this should be doable directly as a python call.
Switchin between bash and python seems less efficient than having more os calls.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, good point!
it’s probably also nicer to just write a Python script instead of inline code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you prefer "inline python" over "Python script that is stored somewhere in the repo"? If script then where is a good place to put it? Under .github/bin or .github/utils .... open for suggestions @IAlibay
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I'll replace it with sleep 60 or something like that for right now but would be generally interested in best practices advice.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
replaced with
sleep 30