Skip to content

fix: partition_email raises KeyError on a multipart/* attachment - #4424

Open
mittalpk wants to merge 2 commits into
Unstructured-IO:mainfrom
mittalpk:fix/partition-email-multipart-attachment-keyerror
Open

fix: partition_email raises KeyError on a multipart/* attachment#4424
mittalpk wants to merge 2 commits into
Unstructured-IO:mainfrom
mittalpk:fix/partition-email-multipart-attachment-keyerror

Conversation

@mittalpk

@mittalpk mittalpk commented Aug 2, 2026

Copy link
Copy Markdown

Fixes #3922

What's wrong

partition_email(..., process_attachments=True) crashes with an uncaught KeyError on emails containing a PGP-signed (or any other multipart/*-typed) attachment — matching the traceback in #3922 exactly.

Root cause: Python's email.contentmanager has no get_content() handler registered for any multipart/* content-type (handlers exist for text/*, application/*, image/*, message/rfc822, etc., but multipart sub-parts are normally consumed via .iter_parts(), not .get_content()). A multipart sub-part can still surface as an "attachment" via EmailMessage.iter_attachments() though — e.g. a PGP/MIME-signed forwarded message (multipart/signed) nested inside a multipart/mixed envelope (the linked testcase.txt scenario), or a multipart/mixed part itself (the exact KeyError: 'multipart/mixed' in the original traceback).

_AttachmentPartitioner._file_bytes (unstructured/partition/email.py) unconditionally calls self._attachment.get_content(), which raises KeyError(content_type) for these cases — and since that call happens before the try/except in _iter_elements() that's meant to gracefully skip unpartitionable attachments, the exception propagates out of the whole partition_email() call instead of being caught.

Fix

_file_bytes now checks self._attachment.get_content_type().startswith("multipart/") and falls back to the part's raw serialized bytes (.as_bytes()) in that case, instead of calling get_content(). This is a root-cause fix at the actual failure point.

I deliberately did not widen EXPECTED_ATTACHMENT_ERRORS (in unstructured/partition/common/__init__.py) to include KeyError as an alternative fix — that tuple's own comment says it's "Intentionally narrow" specifically because a bare KeyError catch there would also silently swallow unrelated bugs elsewhere in partition(). Fixing _file_bytes directly avoids that trade-off entirely.

How was this tested?

  • Reproduced the crash locally with a synthetic .eml (a multipart/mixed email with a nested PGP/MIME-signed forwarded message as an attachment, mirroring the issue's real-world scenario) before writing any fix — confirmed the exact KeyError('multipart/signed').
  • Added test_partition_email_does_not_raise_on_multipart_attachment, using a new fixture example-docs/eml/mime-attach-multipart-signed.eml, following the existing pattern of test_partition_email_silently_skips_attachments_it_cannot_partition (the MP3-attachment test). Confirmed the new test fails against unpatched email.py (git stash) with the exact reported KeyError, and passes after the fix — the email body is still partitioned, and the unpartitionable attachment produces no elements without crashing.
  • Full test_unstructured/partition/test_email.py suite: 72 passed, no regressions.
  • ruff check / ruff format --check — clean.
  • scripts/version-sync.sh -c — clean (no version bump needed; CHANGELOG.md updated under the existing unreleased 0.25.2-dev0 section per the contributing checklist).

Context on prior attempt

A previous PR (#4277) attempted a fix for this same issue but was self-closed by its author after 3 days with no maintainer engagement, and its fixtures never landed — confirmed the bug is still fully live on current main before starting this PR. This PR takes a narrower approach than that one (a single-point fix in _file_bytes, no changes to the shared EXPECTED_ATTACHMENT_ERRORS tuple or to _iter_elements's control flow).

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Shadow auto-approve: would auto-approve. Fixes a KeyError crash when partition_email encounters a multipart/* attachment by falling back to raw serialized bytes; includes a regression test.

Re-trigger cubic

@cragwolfe cragwolfe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Draft review: Unstructured-IO/unstructured PR #4424

Verdict: Request changes

[High] Serialize multipart attachments as standalone documents

unstructured/partition/email.py:443

self._attachment.as_bytes() preserves the part's top-level Content-Disposition: attachment header. When an attachment is named *.eml, auto.partition() routes those serialized bytes back through partition_email(). The serialized part is now the recursive root message, and EmailMessage.get_body() excludes roots whose disposition is attachment, so a valid multipart attachment's body is silently dropped.

I reproduced this with a boundary-bearing multipart/signed attachment: the outer body was returned, but the signed message body was absent. This is introduced by the new serialization path and affects the PGP/MIME forwarded-message case described by the PR.

The added fixture does not exercise that behavior. Its multipart/signed part has no boundary and applies base64 transfer encoding to the multipart entity, so Python parses it as a scalar payload (is_multipart() == False, zero child parts). It proves that the original KeyError no longer escapes, but not that a well-formed multipart attachment is processed correctly.

Please serialize a copy of the multipart part as a standalone document by removing only the copy's top-level Content-Disposition before calling as_bytes(). Preserve the original part for filename metadata and do not mutate the parsed message tree. Add a boundary-bearing multipart/signed regression case that asserts:

  • the outer email body is emitted;
  • the signed attachment body is emitted;
  • the signed body retains filename == "signed-message.eml" and the enclosing attached_to_filename metadata.

The existing malformed fixture can remain as a separate no-crash regression.

(authored by codex)

@eeshsaxena eeshsaxena left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reasonable fix. Since email.contentmanager genuinely has no get_content() handler registered for any multipart/* type, catching that specific case in _AttachmentPartitioner._file_bytes and falling back to the part's raw serialized bytes is the right call, and it lets partition_email() recover the main body instead of the whole call dying on one signed/forwarded sub-part. The regression .eml with a multipart/signed attachment reproduces the original KeyError path, so the test genuinely guards it. Reads correct.

@mittalpk

mittalpk commented Aug 7, 2026

Copy link
Copy Markdown
Author

Good catch. Fixed in 1d1ca7e — the fix now strips Content-Disposition on a copy before serializing, and I rewrote the fixture too since the original had an invalid Content-Transfer-Encoding: base64 on the multipart entity that made Python parse it as a scalar payload with zero child parts, so it could never have exercised this path.

email.contentmanager has no get_content() handler registered for any
multipart/* content-type (handlers exist for text/*, application/*,
image/*, message/rfc822, etc., but multipart sub-parts are normally
consumed via .iter_parts(), not .get_content()). A multipart sub-part
can still surface as an "attachment" via iter_attachments() though --
e.g. a PGP/MIME-signed forwarded message (multipart/signed) nested
inside a multipart/mixed envelope, or a multipart/mixed forwarded
message itself. Resolving such an attachment's bytes in
_AttachmentPartitioner._file_bytes raised KeyError('multipart/signed')
(or 'multipart/mixed'), crashing the entire partition_email() call
instead of processing or skipping just that one attachment.

Fixed by falling back to the MIME part's raw serialized bytes
(as_bytes()) when its content-type is multipart/*, rather than calling
get_content(). This is a root-cause fix at the actual failure point,
not a broadening of the shared, deliberately narrow
EXPECTED_ATTACHMENT_ERRORS tuple (its own comment: "Intentionally
narrow... we do not catch RuntimeError... would otherwise be silently
skipped") -- a bare KeyError there would mask unrelated bugs elsewhere
in partition().

Fixes Unstructured-IO#3922.
Serializing a multipart/* attachment with as_bytes() preserved its
own Content-Disposition: attachment header (that's how it surfaced
as an attachment via iter_attachments() in the first place). Since
EmailMessage.get_body() skips any candidate part -- including a
multipart/* root message -- whose own disposition is "attachment",
re-parsing that serialized part as the root of a new message found
no body at all: the attachment partitioned to zero elements instead
of surfacing its own content (e.g. a PGP-signed forwarded message's
body).

_serialized_multipart_bytes now serializes a deep copy of the part
with that header stripped, leaving the original attachment object
(still needed for its filename) untouched.

Also fixed the existing regression fixture
(mime-attach-multipart-signed.eml): its multipart/signed part had
Content-Transfer-Encoding: base64, which is invalid for a multipart
entity per RFC 2045 -- Python's parser treated it as an opaque
scalar payload with zero child parts, so the fixture could never
have exercised this bug regardless of the fix. Rewrote it as a
genuine boundary-bearing multipart/signed part and extended the
test to assert the attachment's own body and metadata now come
through, not just that no exception is raised.
@mittalpk
mittalpk force-pushed the fix/partition-email-multipart-attachment-keyerror branch from 1d1ca7e to ba4935a Compare August 7, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug/partition_signed_emails

3 participants