feat(python): add pyproject.toml provider support

Add support for pyproject.toml manifest files in the Java client library,
enabling analysis of Python projects using Poetry, uv, or PEP 621-compliant
tools.

- New PythonPyprojectProvider that parses TOML dependency sections and
  generates temporary requirements.txt for dependency tree resolution
- Refactor shared Python infrastructure into PythonProvider base class
- Support PEP 621 [project.dependencies] and Poetry
  [tool.poetry.dependencies] (production deps only)
- Convert Poetry version operators (^ and ~) to PEP 440 ranges
- Handle bare versions, pre-release suffixes, and ignore patterns
- Add license resolution and identification support
- Update README with pyproject.toml documentation
- Comprehensive unit tests for parsing, conversion, and edge cases

Jira-Issue: TC-3851
Assisted-by: Claude Code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: ruromero

README.md

@@ -177,7 +177,7 @@ public class TrustifyExample {
 <li><a href="https://www.java.com/">Java</a> - <a href="https://maven.apache.org/">Maven</a></li>
 <li><a href="https://www.javascript.com//">JavaScript</a> - <a href="https://www.npmjs.com//">Npm</a></li>
 <li><a href="https://go.dev//">Golang</a> - <a href="https://go.dev/blog/using-go-modules//">Go Modules</a></li>
-<li><a href="https://go.dev//">Python</a> - <a href="https://pypi.org/project/pip//">pip Installer</a></li>
+<li><a href="https://www.python.org/">Python</a> - <a href="https://pypi.org/project/pip/">pip Installer</a> (<code>requirements.txt</code>, <code>pyproject.toml</code>)</li>
 <li><a href="https://gradle.org//">Gradle</a> - <a href="https://gradle.org/install//">Gradle Installation</a></li>
 <li><a href="https://www.rust-lang.org/">Rust</a> - <a href="https://doc.rust-lang.org/cargo/">Cargo</a></li>
 
@@ -264,7 +264,7 @@ require (
 </li>
 
 <li>
-<em>Python pip</em> users can add in requirement text a comment with #trustify-da-ignore(or # trustify-da-ignore) to the right of the same artifact to be ignored, for example:
+<em>Python pip</em> users can add in <code>requirements.txt</code> a comment with #trustify-da-ignore(or # trustify-da-ignore) to the right of the same artifact to be ignored, for example:
 
 ```properties
 anyio==3.6.2
@@ -297,6 +297,20 @@ zipp==3.6.0
 ```
 </li>
 
+<li>
+<em>Python pyproject.toml</em> users can add a comment with #trustify-da-ignore next to a dependency in <code>pyproject.toml</code>:
+
+```toml
+[project]
+name = "my-project"
+dependencies = [
+    "requests>=2.28.1",
+    "flask>=2.0",  # trustify-da-ignore
+    "click>=8.0",
+]
+```
+</li>
+
 <li>
 <em>Gradle</em> users can add in build.gradle a comment with //trustify-da-ignore next to the package to be ignored:
 ```build.gradle
@@ -538,7 +552,7 @@ System.setProperty("TRUSTIFY_DA_MVN_LOCAL_REPO", "/home/user/custom-maven-repo")
 ##### Background
 
 In Python pip and in golang go modules package managers ( especially in Python pip) , There is a big chance that for a certain manifest and a given package inside it, the client machine environment has different version installed/resolved
-for that package, which can lead to perform the analysis on the installed packages' versions , instead on the declared versions ( in manifests - that is requirements.txt/go.mod ), and this
+for that package, which can lead to perform the analysis on the installed packages' versions , instead on the declared versions ( in manifests - that is requirements.txt/pyproject.toml/go.mod ), and this
 can cause a confusion for the user in the client consuming the API and leads to inconsistent output ( in THE manifest there is version X For a given Package `A` , and in the analysis report there is another version for the same package `A` - Y).
 
 ##### Usage
@@ -579,16 +593,18 @@ TRUSTIFY_DA_GO_MVS_LOGIC_ENABLED=false
 
 ####  Python Support
 
+Python support works with both `requirements.txt` and `pyproject.toml` manifest files. The `pyproject.toml` provider scans production dependencies from PEP 621 `[project.dependencies]` and Poetry `[tool.poetry.dependencies]` sections. Optional dependencies (`[project.optional-dependencies]`) and Poetry group dependencies (`[tool.poetry.group.*.dependencies]`) are intentionally excluded to focus on runtime dependencies.
+
 By default, Python support assumes that the package is installed using the pip/pip3 binary on the system PATH, or of the customized
 Binaries passed to environment variables. If the package is not installed , then an error will be thrown.
 
-There is an experimental feature of installing the requirement.txt on a virtual env(only python3 or later is supported for this feature) - in this case,
+There is an experimental feature of installing the dependencies on a virtual env(only python3 or later is supported for this feature) - in this case,
 it's important to pass in a path to python3 binary as `TRUSTIFY_DA_PYTHON3_PATH` or instead make sure that python3 is on the system path.
-in such case, You can use that feature by setting environment variable `TRUSTIFY_DA_PYTHON_VIRTUAL_ENV` to true 
+in such case, You can use that feature by setting environment variable `TRUSTIFY_DA_PYTHON_VIRTUAL_ENV` to true
 
 ##### "Best Efforts Installation"
 Since Python pip packages are very sensitive/picky regarding python version changes( every small range of versions is only tailored for a certain python version), I'm introducing this feature, that
-tries to install all packages in requirements.txt onto created virtual environment while **disregarding** versions declared for packages in requirements.txt
+tries to install all packages in the manifest onto created virtual environment while **disregarding** versions declared for packages
 This increasing the chances and the probability a lot that the automatic installation will succeed.
 
 ##### Usage
@@ -693,6 +709,9 @@ java -jar trustify-da-java-client-cli.jar stack /path/to/build.gradle --html
 # Component analysis with JSON output (default)
 java -jar trustify-da-java-client-cli.jar component /path/to/requirements.txt
 
+# Component analysis for pyproject.toml
+java -jar trustify-da-java-client-cli.jar component /path/to/pyproject.toml
+
 # Component analysis with summary
 java -jar trustify-da-java-client-cli.jar component /path/to/go.mod --summary
 

src/main/java/io/github/guacsec/trustifyda/providers/PythonPipProvider.java

@@ -16,196 +16,42 @@
  */
 package io.github.guacsec.trustifyda.providers;
 
-import static io.github.guacsec.trustifyda.impl.ExhortApi.debugLoggingIsNeeded;
-
-import com.fasterxml.jackson.core.JsonProcessingException;
-import com.github.packageurl.MalformedPackageURLException;
 import com.github.packageurl.PackageURL;
-import io.github.guacsec.trustifyda.Api;
-import io.github.guacsec.trustifyda.Provider;
-import io.github.guacsec.trustifyda.license.LicenseUtils;
-import io.github.guacsec.trustifyda.logging.LoggersFactory;
-import io.github.guacsec.trustifyda.sbom.Sbom;
-import io.github.guacsec.trustifyda.sbom.SbomFactory;
-import io.github.guacsec.trustifyda.tools.Ecosystem;
-import io.github.guacsec.trustifyda.tools.Operations;
-import io.github.guacsec.trustifyda.utils.Environment;
-import io.github.guacsec.trustifyda.utils.IgnorePatternDetector;
 import io.github.guacsec.trustifyda.utils.PythonControllerBase;
-import io.github.guacsec.trustifyda.utils.PythonControllerRealEnv;
-import io.github.guacsec.trustifyda.utils.PythonControllerVirtualEnv;
-import java.io.IOException;
-import java.nio.charset.StandardCharsets;
-import java.nio.file.Files;
 import java.nio.file.Path;
 import java.util.Arrays;
-import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.logging.Logger;
 import java.util.stream.Collectors;
 
-public final class PythonPipProvider extends Provider {
-
-  private static final Logger log = LoggersFactory.getLogger(PythonPipProvider.class.getName());
-  private static final String DEFAULT_PIP_ROOT_COMPONENT_NAME = "default-pip-root";
-  private static final String DEFAULT_PIP_ROOT_COMPONENT_VERSION = "0.0.0";
-
-  public void setPythonController(PythonControllerBase pythonController) {
-    this.pythonController = pythonController;
-  }
-
-  private PythonControllerBase pythonController;
+public final class PythonPipProvider extends PythonProvider {
 
   public PythonPipProvider(Path manifest) {
-    super(Ecosystem.Type.PYTHON, manifest);
+    super(manifest);
   }
 
   @Override
-  public String readLicenseFromManifest() {
-    return LicenseUtils.readLicenseFile(manifest);
+  protected Path getRequirementsPath() {
+    return manifest;
   }
 
   @Override
-  public Content provideStack() throws IOException {
-    PythonControllerBase pythonController = getPythonController();
-    List<Map<String, Object>> dependencies =
-        pythonController.getDependencies(manifest.toString(), true);
-    printDependenciesTree(dependencies);
-    Sbom sbom = SbomFactory.newInstance(Sbom.BelongingCondition.PURL, "sensitive");
-    sbom.addRoot(
-        toPurl(DEFAULT_PIP_ROOT_COMPONENT_NAME, DEFAULT_PIP_ROOT_COMPONENT_VERSION),
-        readLicenseFromManifest());
-    for (Map<String, Object> component : dependencies) {
-      addAllDependencies(sbom.getRoot(), component, sbom);
-    }
-    byte[] requirementsFile = Files.readAllBytes(manifest);
-    handleIgnoredDependencies(new String(requirementsFile), sbom);
-    return new Content(
-        sbom.getAsJsonString().getBytes(StandardCharsets.UTF_8), Api.CYCLONEDX_MEDIA_TYPE);
-  }
-
-  private void addAllDependencies(PackageURL source, Map<String, Object> component, Sbom sbom) {
-
-    PackageURL packageURL =
-        toPurl((String) component.get("name"), (String) component.get("version"));
-    sbom.addDependency(source, packageURL, null);
-
-    List<Map<String, Object>> directDeps =
-        (List<Map<String, Object>>) component.get("dependencies");
-    if (directDeps != null) {
-      for (Map<String, Object> dep : directDeps) {
-        addAllDependencies(packageURL, dep, sbom);
-      }
-    }
+  protected void cleanupRequirementsPath(Path requirementsPath) {
+    // No cleanup needed — the manifest is the requirements file itself.
   }
 
   @Override
-  public Content provideComponent() throws IOException {
-    PythonControllerBase pythonController = getPythonController();
-    List<Map<String, Object>> dependencies =
-        pythonController.getDependencies(manifest.toString(), false);
-    printDependenciesTree(dependencies);
-    Sbom sbom = SbomFactory.newInstance();
-    sbom.addRoot(
-        toPurl(DEFAULT_PIP_ROOT_COMPONENT_NAME, DEFAULT_PIP_ROOT_COMPONENT_VERSION),
-        readLicenseFromManifest());
-    dependencies.forEach(
-        (component) ->
-            sbom.addDependency(
-                sbom.getRoot(),
-                toPurl((String) component.get("name"), (String) component.get("version")),
-                null));
-
-    var manifestContent = Files.readString(manifest);
-    handleIgnoredDependencies(manifestContent, sbom);
-    return new Content(
-        sbom.getAsJsonString().getBytes(StandardCharsets.UTF_8), Api.CYCLONEDX_MEDIA_TYPE);
+  protected Set<PackageURL> getIgnoredDependencies(String manifestContent) {
+    String[] lines = manifestContent.split(System.lineSeparator());
+    return Arrays.stream(lines)
+        .filter(this::containsIgnorePattern)
+        .map(PythonPipProvider::extractDepFull)
+        .map(this::splitToNameVersion)
+        .map(dep -> toPurl(dep[0], dep[1]))
+        .collect(Collectors.toSet());
   }
 
-  private void printDependenciesTree(List<Map<String, Object>> dependencies)
-      throws JsonProcessingException {
-    if (debugLoggingIsNeeded()) {
-      String pythonControllerTree =
-          objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(dependencies);
-      log.info(
-          String.format(
-              "Python Generated Dependency Tree in Json Format: %s %s %s",
-              System.lineSeparator(), pythonControllerTree, System.lineSeparator()));
-    }
-  }
-
-  private void handleIgnoredDependencies(String manifestContent, Sbom sbom) {
-    Set<PackageURL> ignoredDeps = getIgnoredDependencies(manifestContent);
-    Set<String> ignoredDepsVersions =
-        ignoredDeps.stream()
-            .filter(dep -> !dep.getVersion().trim().equals("*"))
-            .map(PackageURL::getCoordinates)
-            .collect(Collectors.toSet());
-    Set<String> ignoredDepsNoVersions =
-        ignoredDeps.stream()
-            .filter(dep -> dep.getVersion().trim().equals("*"))
-            .map(PackageURL::getCoordinates)
-            .collect(Collectors.toSet());
-
-    // filter out by name only from sbom all exhortignore dependencies that their version will be
-    // resolved by pip.
-    sbom.setBelongingCriteriaBinaryAlgorithm(Sbom.BelongingCondition.NAME);
-    sbom.filterIgnoredDeps(ignoredDepsNoVersions);
-    boolean matchManifestVersions = Environment.getBoolean(PROP_MATCH_MANIFEST_VERSIONS, true);
-    // filter out by purl from sbom all exhortignore dependencies that their version hardcoded in
-    // requirements.txt -
-    // in case all versions in manifest matching installed versions of packages in environment.
-    if (matchManifestVersions) {
-      sbom.setBelongingCriteriaBinaryAlgorithm(Sbom.BelongingCondition.PURL);
-      sbom.filterIgnoredDeps(ignoredDepsVersions);
-    } else {
-      // in case version mismatch is possible (MATCH_MANIFEST_VERSIONS=false) , need to parse the
-      // name of package
-      // from the purl, and remove the package name from sbom according to name only
-      Set<String> deps =
-          ignoredDepsVersions.stream()
-              .map(
-                  purlString -> {
-                    try {
-                      return new PackageURL(purlString).getName();
-                    } catch (MalformedPackageURLException e) {
-                      throw new RuntimeException(e);
-                    }
-                  })
-              .collect(Collectors.toSet());
-      sbom.setBelongingCriteriaBinaryAlgorithm(Sbom.BelongingCondition.NAME);
-      sbom.filterIgnoredDeps(deps);
-    }
-  }
-
-  /**
-   * Checks if a text line contains a Python pip ignore pattern. Handles both '#exhortignore' and
-   * '#trustify-da-ignore' with optional spacing.
-   *
-   * @param line the line to check
-   * @return true if the line contains a Python pip ignore pattern
-   */
-  private boolean containsPythonIgnorePattern(String line) {
-    return line.contains("#" + IgnorePatternDetector.IGNORE_PATTERN)
-        || line.contains("# " + IgnorePatternDetector.IGNORE_PATTERN)
-        || line.contains("#" + IgnorePatternDetector.LEGACY_IGNORE_PATTERN)
-        || line.contains("# " + IgnorePatternDetector.LEGACY_IGNORE_PATTERN);
-  }
-
-  private Set<PackageURL> getIgnoredDependencies(String requirementsDeps) {
-
-    String[] requirementsLines = requirementsDeps.split(System.lineSeparator());
-    Set<PackageURL> collected =
-        Arrays.stream(requirementsLines)
-            .filter(this::containsPythonIgnorePattern)
-            .map(PythonPipProvider::extractDepFull)
-            .map(this::splitToNameVersion)
-            .map(dep -> toPurl(dep[0], dep[1]))
-            //      .map(packageURL -> packageURL.getCoordinates())
-            .collect(Collectors.toSet());
-
-    return collected;
+  private static String extractDepFull(String requirementLine) {
+    return requirementLine.substring(0, requirementLine.indexOf("#")).trim();
   }
 
   private String[] splitToNameVersion(String nameVersion) {
@@ -219,91 +65,4 @@ private String[] splitToNameVersion(String nameVersion) {
     }
     return result;
   }
-
-  private static String extractDepFull(String requirementLine) {
-    return requirementLine.substring(0, requirementLine.indexOf("#")).trim();
-  }
-
-  private PackageURL toPurl(String name, String version) {
-
-    try {
-      return new PackageURL(Ecosystem.Type.PYTHON.getType(), null, name, version, null, null);
-    } catch (MalformedPackageURLException e) {
-      throw new RuntimeException(e);
-    }
-  }
-
-  private PythonControllerBase getPythonController() {
-    String pythonPipBinaries;
-    boolean useVirtualPythonEnv;
-    if (!Environment.get(PythonControllerBase.PROP_TRUSTIFY_DA_PIP_SHOW, "").trim().isEmpty()
-        && !Environment.get(PythonControllerBase.PROP_TRUSTIFY_DA_PIP_FREEZE, "")
-            .trim()
-            .isEmpty()) {
-      pythonPipBinaries = "python;;pip";
-      useVirtualPythonEnv = false;
-    } else {
-      pythonPipBinaries = getExecutable("python", "--version");
-      useVirtualPythonEnv =
-          Environment.getBoolean(PythonControllerBase.PROP_TRUSTIFY_DA_PYTHON_VIRTUAL_ENV, false);
-    }
-
-    String[] parts = pythonPipBinaries.split(";;");
-    var python = parts[0];
-    var pip = parts[1];
-    PythonControllerBase pythonController;
-    if (this.pythonController == null) {
-      if (useVirtualPythonEnv) {
-        pythonController = new PythonControllerVirtualEnv(python);
-      } else {
-        pythonController = new PythonControllerRealEnv(python, pip);
-      }
-    } else {
-      pythonController = this.pythonController;
-    }
-    return pythonController;
-  }
-
-  private String getExecutable(String command, String args) {
-    String python = Operations.getCustomPathOrElse("python3");
-    String pip = Operations.getCustomPathOrElse("pip3");
-    try {
-      Operations.runProcess(python, args);
-      Operations.runProcess(pip, args);
-    } catch (Exception e) {
-      python = Operations.getCustomPathOrElse("python");
-      pip = Operations.getCustomPathOrElse("pip");
-      try {
-        Process process = new ProcessBuilder(command, args).redirectErrorStream(true).start();
-        int exitCode = process.waitFor();
-        if (exitCode != 0) {
-          throw new IOException(
-              "Python executable found, but it exited with error code " + exitCode);
-        }
-      } catch (IOException | InterruptedException ex) {
-        throw new RuntimeException(
-            String.format(
-                "Unable to find or run Python executable '%s'. Please ensure Python is installed"
-                    + " and available in your PATH.",
-                command),
-            ex);
-      }
-
-      try {
-        Process process = new ProcessBuilder("pip", args).redirectErrorStream(true).start();
-        int exitCode = process.waitFor();
-        if (exitCode != 0) {
-          throw new IOException("Pip executable found, but it exited with error code " + exitCode);
-        }
-      } catch (IOException | InterruptedException ex) {
-        throw new RuntimeException(
-            String.format(
-                "Unable to find or run Pip executable '%s'. Please ensure Pip is installed and"
-                    + " available in your PATH.",
-                command),
-            ex);
-      }
-    }
-    return String.format("%s;;%s", python, pip);
-  }
 }