Add forwarded email extraction to email_import module

[adulau]

Motivation

• Forwarded messages (inline Outlook-style forwards and message/rfc822 attachments) were not being emitted as dependent email objects, reducing analyst visibility and matching behavior in mail_to_misp.
• Improve the email importer by reusing heuristics from mail_to_misp to detect inline-forward blocks and attached .eml forwards.

Description

• Add new module config extract_forwarded_emails (enabled by default) and wire it into dict_handler to produce dependent email objects for forwards.
• Skip message/rfc822 parts in the regular attachment path and add helper functions iter_attached_email_bytes, iter_decoded_bodies, find_inline_forwarded_email_bytes, build_forwarded_message, add_forwarded_email_object, and iter_file_attachments to extract forwards robustly.
• Use message_from_bytes with policy for parsing constructed RFC822 snippets and extend HTMLTextParser to preserve structural line breaks to improve forwarded-header detection in HTML bodies.
• Add regression tests in tests/test_email_import_forwarded.py covering plain-text inline forwards, HTML inline forwards, and attached .eml forwards.

Testing

• Ran pytest for the new regression suite with python -m pytest tests/test_email_import_forwarded.py -q and all tests passed (3 passed).
• Verified module syntax with python -m py_compile misp_modules/modules/import_mod/email_import.py and style checks with python -m ruff check misp_modules/modules/import_mod/email_import.py tests/test_email_import_forwarded.py, both of which succeeded.
• Existing legacy email_headers test is still skipped (Need Rewrite) and was not modified by this change.

---

Codex Task

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 58cbf1c89d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Do not drop rfc822 attachments when extraction is off

When callers set extract_forwarded_emails to false to avoid creating dependent email objects, this unconditional skip removes message/rfc822 attachments from the normal attachment path and the later extraction block is not run, so attached .eml forwards disappear from the import results entirely. The skip should be tied to successful forwarded-email extraction or fall back to emitting the attachment as a file object when extraction is disabled.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Map localized From headers before requiring from

When an inline forwarded block starts with De: or Von: (which the start detector explicitly accepts), the builder still only stores English header names and later requires a stored from header, so those French/German forwards are discarded even if the rest of the header block is present. Normalize de/von to from here, or remove them from the detector if they are not intended to work.

Useful? React with 👍 / 👎.