Detect and collect dependencies from gradle cache

[pombredanne]

Gradle maintains a build cache with a trove of package origins. This may not be super useful for a regular codebase scan, but is a great way to collect PURLs for indexing prior to matching.

There is a .module JSON file store in a path like this:
gradle|.gradle/modules-2/files-2.1/maven dotted groupid, like "io.spring.gradle"/maven artifactid like "dependency-management-plugin"/version like 1.1.3/SHA1 of artifact like 3209385654a7e661d68de95a5ea8fc11d8ce015e/<name>-<version>.module like in dependency-management-plugin-1.1.3.module

And that looks like that:

{
  "formatVersion": "1.1",
  "component": {
    "group": "com.google.android.material",
    "module": "material",
    "version": "1.9.0",
    "attributes": {
      "org.gradle.status": "release"
    }
  },
  "createdBy": {
    "gradle": {
      "version": "7.3.3"
    }
  },
  "variants": [
    {
      "name": "releaseVariantReleaseApiPublication",
      "attributes": {
        "org.gradle.category": "library",
        "org.gradle.dependency.bundling": "external",
        "org.gradle.libraryelements": "aar",
        "org.gradle.usage": "java-api"
      },
      "dependencies": [
        {
          "group": "androidx.annotation",
          "module": "annotation",
          "version": {
            "requires": "1.2.0"
          }
        },
        {
          "group": "androidx.appcompat",
          "module": "appcompat",
          "version": {
            "requires": "1.5.0"
          }
        },
        {
          "group": "androidx.cardview",
          "module": "cardview",
          "version": {
            "requires": "1.0.0"
          }
        },
        {
          "group": "androidx.coordinatorlayout",
          "module": "coordinatorlayout",
          "version": {
            "requires": "1.1.0"
          }
        },
        {
          "group": "androidx.constraintlayout",
          "module": "constraintlayout",
          "version": {
            "requires": "2.0.1"
          }
        },
........
        }
      ],
      "files": [
        {
          "name": "material-1.9.0.aar",
          "url": "material-1.9.0.aar",
          "size": 2202178,
          "sha512": "7630aacb9e3073b2064397ed080b8d5bf7db06ba2022d6c927e05b7d53c5787d8153cd4a0360c36d52a79a7361e0b87f9d5fe856caf5ce5e7a89a9f30d9c7cce",
          "sha256": "6cc2359979269e4d9eddce7d84682d2bb06a35a14edce806bf0da6e8d4d31806",
          "sha1": "08f4a93a381be223a5bbaacd46eaab92381ab6a8",
          "md5": "3287103cfb083fb998a35ef8a1983c58"
        }
      ]
    },
    {
      "name": "releaseVariantReleaseRuntimePublication",
      "attributes": {
        "org.gradle.category": "library",
        "org.gradle.dependency.bundling": "external",
        "org.gradle.libraryelements": "aar",
        "org.gradle.usage": "java-runtime"
      },
      "dependencies": [
        {
          "group": "com.google.errorprone",
          "module": "error_prone_annotations",
          "version": {
            "requires": "2.15.0"
          }
        },
        {
          "group": "androidx.annotation",
          "module": "annotation",
          "version": {
            "requires": "1.2.0"
          }
        },
        {
.................

See also:

• https://docs.gradle.org/current/userguide/build_cache.html
• https://docs.gradle.org/current/userguide/build_cache_use_cases.html
• https://stackoverflow.com/questions/23025433/how-to-clear-gradle-cache

See examples of these at:

• https://github.com/msdev27/CashCard/blob/799e8df1f9e7a9339d1bfe812a2ddc5bde1c3668/cashcard/gradle/caches/modules-2/files-2.1/io.spring.gradle/dependency-management-plugin/1.1.3/3209385654a7e661d68de95a5ea8fc11d8ce015e/dependency-management-plugin-1.1.3.module#L4
• https://github.com/goyoalucard/Farmacia/blob/627268aa05cfd41e274211da0091a7feb15a82cd/.gradle/7.5.1/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-core-jvm/1.6.1/91d3217f26f78ef3b56e06b34b1233dab25c713/kotlinx-coroutines-core-jvm-1.6.1.module#L4
• https://github.com/liferay/liferay-binaries-cache-2020/blob/47003439ba0019bdbdbe2a15433d5dd3db5246c6/.gradle/caches/modules-2/files-2.1/com.liferay/com.liferay.gradle.plugins.source.formatter/5.2.69/e6e7283776c03991cfebb692ebd7d612de0852ff/com.liferay.gradle.plugins.source.formatter-5.2.69.module#L4

[Arya-Hari]

Hello @pombredanne, I would like to work on this issue. I do have some questions regarding how this task should be executed.

• Would building a Python script that parses through the JSON file, detects the dependencies, extracts and saves them as output be sufficient?
• If so, in what format should the output be in? That is, should it be saved to a text file or another JSON file?
• Lastly, how should the output information be structured? What information regarding the dependencies must be collected? Should "group", "module" and "version" of each of these dependencies be collected?

Thank you!

@AyanSinhaMahapatra @pombredanne can you assign this to me

[BRGOVIND]

“Hi, I’d like to work on this issue. Is it still available?”

[Syimyk43]

Hi, I am a GSoC 2026 applicant interested in the ScanCode.io file system tree view project. I have Django experience and I am currently looking at good first issues to contribute. Could you guide me on which issues would best prepare me for this project? Thank you!
@AyanSinhaMahapatra

[kumarasantosh]

Hi @pombredanne, I'm Santosh, a GSoC 2026 applicant working
on the "Expand Package Format Support" project. I've already
opened PRs #4803, #4804, and #4805 for NuGet deps.json,
Conda environment.yml, and Cargo deny.toml.

I'd like to work on this issue as well. I'll open a PR soon.