Merge branch 'main' into main

Author: vaidikcode

.github/ISSUE_TEMPLATE/release-checklist.md

@@ -7,41 +7,42 @@ assignees: ''
 
 ---
 
-- [ ] Confirm anything outstanding for release with other maintainers on #besu-release in Discord
+- [ ] Confirm at least 24 hours prior anything outstanding for release with other maintainers on #besu-release in Discord
 - [ ] Update changelog if necessary, and merge a PR for it to main
   - [ ] Notify maintainers about updating changelog for in-flight PRs 
 - [ ] Optional: for hotfixes, create a release branch and cherry-pick, e.g. `release-<version>-hotfix`
   - [ ] Optional: for hotfixes, create a PR into main from the hotfix branch to see the CI checks pass
 - [ ] On the appropriate branch/commit, create a calver tag for the release candidate, format example: `24.4.0-RC1`
-  - [ ] git tag 24.4.0-RC1
-  - [ ] git push upstream 24.4.0-RC1
+  - [ ] `git tag 24.4.0-RC1`
+  - [ ] `git push upstream 24.4.0-RC1`
 - [ ] Sign-off with team; announce the tag in #besu-release in Discord
   - [ ] Targeting this tag for the burn-in: https://github.com/hyperledger/besu/releases/tag/24.4.0-RC1
 - [ ] Consensys staff start burn-in using this tag
 - [ ] Seek sign off for burn-in
   - [ ] Pass? Go ahead and complete the release process
   - [ ] Fail? Put a message in #besu-release in Discord indicating the release will be aborted because it failed burn-in 
 - [ ] Optional: Perform a dry run with https://github.com/consensys/protocols-release-sandbox to test the workflows
-  - [ ] Sync fork
-  - [ ] git checkout <sha of 24.4.0-RC1>
-  - [ ] git tag 24.4.0
-  - [ ] git push origin 24.4.0
+  - [ ] Sync fork in github
+  - [ ] `git checkout <sha of 24.4.0-RC1>`
+  - [ ] `git tag 24.4.0`
+  - [ ] `git push <your remote name> 24.4.0`
   - [ ] Manually run https://github.com/Consensys/protocols-release-sandbox/actions/workflows/draft-release.yml using `main` branch and `24.4.0` tag
 - [ ] Back on besu, using the same git sha as 24.4.0-RC1, create a calver tag for the FULL RELEASE, example format `24.4.0`
-  - [ ] git checkout 24.4.0-RC1
-  - [ ] git tag 24.4.0
-  - [ ] git push upstream 24.4.0
+  - [ ] `git checkout 24.4.0-RC1`
+  - [ ] `git tag 24.4.0`
+  - [ ] `git push upstream 24.4.0`
 - [ ] Manually run https://github.com/hyperledger/besu/actions/workflows/draft-release.yml using `main` branch` and the FULL RELEASE tag name, i.e. `24.4.0`. Note, this workflow should always be run from `main` branch (hotfix tags will still be released even if they were created based on another branch)
     - publishes artefacts and version-specific docker tags but does not fully publish the GitHub release so subscribers are not yet notified
 - [ ] Check all draft-release workflow jobs went green
 - [ ] Check binary SHAs are correct on the release page
+- [ ] Check artifacts exist in https://hyperledger.jfrog.io/ui/repos/tree/General/besu-maven
 - [ ] Update release notes in the GitHub draft release, save draft and sign-off with team
 - [ ] Publish draft release ensuring it is marked as latest release (if appropriate)
     - this is now public and notifies subscribed users
     - makes the release "latest" in github
     - publishes the docker `latest` tag variants
-- [ ] Create homebrew release using [update-version workflow](https://github.com/hyperledger/homebrew-besu/actions/workflows/update-version.yml)
+- [ ] Create homebrew release PR using [update-version workflow](https://github.com/hyperledger/homebrew-besu/actions/workflows/update-version.yml)
   - If the PR has not been automatically created, create the PR manually using the created branch `update-<version>`
-  - Run commands `brew tap hyperledger/besu && brew install besu` on MacOSX and verify latest version has been installed
+- [ ] Verify homebrew release once the PR has merged using `brew tap hyperledger/besu && brew install besu` on MacOSX to verify latest version has been installed
 - [ ] Delete the burn-in nodes (unless required for further analysis eg performance)
 - [ ] Social announcements

.github/pull_request_template.md

@@ -1,3 +1,4 @@
+
 ## PR description
 
 ## Fixed Issue(s)
@@ -14,6 +15,7 @@
 
 ### Locally, you can run these tests to catch failures early:
 
+- [ ] spotless: `./gradlew spotlessApply`
 - [ ] unit tests: `./gradlew build`
 - [ ] acceptance tests: `./gradlew acceptanceTest`
 - [ ] integration tests: `./gradlew integrationTest`

.github/workflows/acceptance-tests.yml

@@ -1,17 +1,19 @@
 name: acceptance-tests
 on:
   workflow_dispatch:
+  merge_group:
   pull_request:
     branches:
       - main
       - release-*
+      - verkle
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
   cancel-in-progress: true
 
 env:
-  GRADLE_OPTS: "-Xmx7g"
+  GRADLE_OPTS: "-Xmx1g"
   total-runners: 14
 
 jobs:
@@ -77,7 +79,7 @@ jobs:
           name: test-args-${{ matrix.runner_index }}.txt
           path: '*.txt'
       - name: run acceptance tests
-        run: ./gradlew acceptanceTestNotPrivacy `cat gradleArgs.txt` -Dorg.gradle.parallel=true -Dorg.gradle.caching=true
+        run: ./gradlew --max-workers 1 acceptanceTestNotPrivacy `cat gradleArgs.txt` -Dorg.gradle.caching=true
       - name: Remove downloaded test results
         run: rm -rf tmp/junit-xml-reports-downloaded
       - name: Upload Acceptance Test Results

.github/workflows/draft-release.yml

@@ -396,3 +396,23 @@ jobs:
           ARTIFACTORY_USER: ${{ secrets.BESU_ARTIFACTORY_USER }}
           ARTIFACTORY_KEY: ${{ secrets.BESU_ARTIFACTORY_TOKEN }}
         run: ./gradlew -Prelease.releaseVersion=${{ env.RELEASE_VERSION }} -Pversion=${{env.RELEASE_VERSION}} artifactoryPublish
+
+  verify_artifactory:
+    runs-on: ubuntu-22.04
+    needs: [artifactory, validate, test-linux, test-windows]
+    steps:
+      - name: checkout
+        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
+        with:
+          ref: ${{ env.RELEASE_VERSION }}
+
+      # actions/setup-python@5.3 
+      - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b
+        with:
+          python-version: '3.13'
+
+      - name: Install dependencies
+        run: pip install requests argparse
+
+      - name: Run the script
+        run: python3 .github/workflows/verify_artifacts.py --besu_version="${{ needs.validate.outputs.release_version }}"

.github/workflows/integration-tests.yml

@@ -1,10 +1,12 @@
 name: integration-tests
 on:
   workflow_dispatch:
+  merge_group:
   pull_request:
     branches:
       - main
       - release-*
+      - verkle
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}

.github/workflows/pre-review.yml

@@ -1,18 +1,20 @@
 name: pre-review
 
 on:
+  merge_group:
   pull_request:
     branches:
       - main
       - release-*
+      - verkle
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
   cancel-in-progress: true
 
 env:
   GRADLE_OPTS: "-Xmx6g -Dorg.gradle.parallel=true"
-  total-runners: 10
+  total-runners: 14
 
 jobs:
   repolint:
@@ -83,7 +85,7 @@ jobs:
     strategy:
       fail-fast: true
       matrix:
-        runner_index: [0,1,2,3,4,5,6,7,8,9]
+        runner_index: [0,1,2,3,4,5,6,7,8,9,10,11,12,13]
     steps:
       - name: Checkout Repo
         uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11

.github/workflows/reference-tests.yml

@@ -1,14 +1,16 @@
 name: reference-tests
 on:
   workflow_dispatch:
+  merge_group:
   pull_request:
     branches:
       - main
       - release-*
+      - verkle
 
 env:
   GRADLE_OPTS: "-Xmx6g -Dorg.gradle.daemon=false -Dorg.gradle.parallel=true -Dorg.gradle.caching=true"
-  total-runners: 8
+  total-runners: 4
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
@@ -24,7 +26,7 @@ jobs:
     strategy:
       fail-fast: true
       matrix:
-        runner_index: [1,2,3,4,5,6,7,8]
+        runner_index: [1,2,3,4]
     steps:
       - name: Checkout Repo
         uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11

.github/workflows/sonarcloud.yml

@@ -17,6 +17,8 @@ jobs:
     steps:
       - name: checkout
         uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
+        with:
+          fetch-depth: 0  # Shallow clones should be disabled for a better relevancy of analysis
       - name: Set up Java
         uses: actions/setup-java@387ac29b308b003ca37ba93a6cab5eb57c8f5f93
         with:

.github/workflows/verify_artifacts.py

@@ -0,0 +1,55 @@
+import requests
+import argparse
+
+
+def create_artifact_paths(version):
+    artifacts_base_path = "https://hyperledger.jfrog.io/hyperledger/besu-maven/org/hyperledger/besu"
+    # add to this list here to update the list of artifacts to check 
+    artifacts = [
+        # besu/evm
+        f"{artifacts_base_path}/evm/{version}/evm-{version}.module",
+        f"{artifacts_base_path}/evm/{version}/evm-{version}.pom",
+        f"{artifacts_base_path}/evm/{version}/evm-{version}.jar",
+        # besu/plugin-api
+        f"{artifacts_base_path}/plugin-api/{version}/plugin-api-{version}.module",
+        f"{artifacts_base_path}/plugin-api/{version}/plugin-api-{version}.pom",
+        f"{artifacts_base_path}/plugin-api/{version}/plugin-api-{version}.jar",
+        # besu/metrics-core
+        f"{artifacts_base_path}/internal/metrics-core/{version}/metrics-core-{version}.module",
+        f"{artifacts_base_path}/internal/metrics-core/{version}/metrics-core-{version}.pom",
+        f"{artifacts_base_path}/internal/metrics-core/{version}/metrics-core-{version}.jar",
+        # besu/internal/core
+        f"{artifacts_base_path}/internal/core/{version}/core-{version}.module",
+        f"{artifacts_base_path}/internal/core/{version}/core-{version}.pom",
+        f"{artifacts_base_path}/internal/core/{version}/core-{version}.jar",
+        # besu/internal/config
+        f"{artifacts_base_path}/internal/config/{version}/config-{version}.module",
+        f"{artifacts_base_path}/internal/config/{version}/config-{version}.pom",
+        f"{artifacts_base_path}/internal/config/{version}/config-{version}.jar",
+        # bom
+        f"{artifacts_base_path}/bom/{version}/bom-{version}.module",
+        f"{artifacts_base_path}/bom/{version}/bom-{version}.pom",
+    ]
+    return artifacts
+
+
+
+def check_url(url):
+    print(f"Checking artifact at: {url}")
+    r = requests.head(url)
+    if (r.status_code != 200):
+        raise Exception(f"Sorry, No artifact found at '{url}' !!!") 
+
+def main():
+    parser = argparse.ArgumentParser(description='Check besu artifacts')
+    parser.add_argument('--besu_version', action="store", dest='besu_version', default="")
+    args = parser.parse_args()
+    print(args.besu_version)
+
+    artifacts = create_artifact_paths(args.besu_version)
+    print(artifacts)
+    for url in artifacts:
+        check_url(url)
+
+if __name__ == "__main__":
+    main()