feat: remove poppler-utils dependency by using pypdf for page counts

[willstott101]

To make our containers smaller and have fewer CVEs detected - I've been removing dependencies we don't use from our unstructured deployment. poppler-utils on debian trixie for instance has some CVEs associated with it.

Once I realised this project has almost entirely moved away from pdf2image in #4185 I thought I might as-well contribute finishing the job.

I have made some effort to ensure this is not a regression in performance or behaviour. From limited testing I think this is faster for small PDFs (probably due to no sub-process startup) and on-par for larger PDFs (thanks to opening the file and only reading the amount we need).

This code in this PR is written by Cursor with my supervision - I am heavily relying on the test suite to validate behaviour. If you are aware of any corner cases that aren't captured by the existing suite in terms of the sorts of weird PDFs that this PR might trip up on lmk and I can try and expand it.

[cubic-dev-ai]

1 issue found across 21 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="unstructured/partition/pdf_image/pypdf_utils.py">

<violation number="1" location="unstructured/partition/pdf_image/pypdf_utils.py:13">
P2: The new `pdf_page_count()` reimplements page counting by reading the catalog `/Pages /Count` directly instead of using `PdfReader.get_num_pages()` (which the replaced code called). This is slightly more fragile than the pypdf API it replaces: if a PDF's root `/Pages` dictionary omits `/Count` (seen in some generated/malformed PDFs), accessing `root_object["/Pages"]["/Count"]` raises `KeyError`, and `int()` raises `ValueError` on a non-numeric value. It also trusts the declared `/Count` even when it disagrees with the actual page tree, which is exactly the class of "weird PDF" the PR asks about. Since `pdf_page_count()` feeds the chunking loop in `convert_pdf_to_images` (`range(1, total_pages + 1, chunk_size)`) and page-count checks in `pdf.py`, a wrong or missing value either truncates pages or triggers rendering of non-existent pages. Recommend adding a fallback (or delegating to `get_num_pages()`) so behavior degrades gracefully instead of raising.</violation>
</file>

<sub>Shadow auto-approve: would not auto-approve because issues were found.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: The new pdf_page_count() reimplements page counting by reading the catalog /Pages /Count directly instead of using PdfReader.get_num_pages() (which the replaced code called). This is slightly more fragile than the pypdf API it replaces: if a PDF's root /Pages dictionary omits /Count (seen in some generated/malformed PDFs), accessing root_object["/Pages"]["/Count"] raises KeyError, and int() raises ValueError on a non-numeric value. It also trusts the declared /Count even when it disagrees with the actual page tree, which is exactly the class of "weird PDF" the PR asks about. Since pdf_page_count() feeds the chunking loop in convert_pdf_to_images (range(1, total_pages + 1, chunk_size)) and page-count checks in pdf.py, a wrong or missing value either truncates pages or triggers rendering of non-existent pages. Recommend adding a fallback (or delegating to get_num_pages()) so behavior degrades gracefully instead of raising.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At unstructured/partition/pdf_image/pypdf_utils.py, line 13:

<comment>The new `pdf_page_count()` reimplements page counting by reading the catalog `/Pages /Count` directly instead of using `PdfReader.get_num_pages()` (which the replaced code called). This is slightly more fragile than the pypdf API it replaces: if a PDF's root `/Pages` dictionary omits `/Count` (seen in some generated/malformed PDFs), accessing `root_object["/Pages"]["/Count"]` raises `KeyError`, and `int()` raises `ValueError` on a non-numeric value. It also trusts the declared `/Count` even when it disagrees with the actual page tree, which is exactly the class of "weird PDF" the PR asks about. Since `pdf_page_count()` feeds the chunking loop in `convert_pdf_to_images` (`range(1, total_pages + 1, chunk_size)`) and page-count checks in `pdf.py`, a wrong or missing value either truncates pages or triggers rendering of non-existent pages. Recommend adding a fallback (or delegating to `get_num_pages()`) so behavior degrades gracefully instead of raising.</comment>

<file context>
@@ -2,6 +2,15 @@
+
+    Matches poppler ``pdfinfo`` ``Pages`` (PDF catalog ``/Pages`` ``/Count``).
+    """
+    return int(reader.root_object["/Pages"]["/Count"])
 
 
</file context>

Suggested change

	return int(reader.root_object["/Pages"]["/Count"])
	def get_page_data(fp: BinaryIO, page_number: int):
	def pdf_page_count(reader: PdfReader) -> int:
	"""Return the number of pages, falling back to page-tree traversal if /Count is missing."""
	try:
	return int(reader.root_object["/Pages"]["/Count"])
	except (KeyError, ValueError):
	return len(list(reader.pages))