Skip to content

Preserve bracketed prefixes when stripping email subject markers #792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
adulau wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Preserve bracketed prefixes when stripping email subject markers #792

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion misp_modules/modules/import_mod/email_import.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,6 @@ def dict_handler(request: dict):

SUBJECT_MARKER_RE = re.compile(r"""
^\s*
(?:(?:\[[^\]]+\]\s*)*)
(?:
(?:re|fw|fwd|wg|aw|sv|tr|rv|vs)
Comment on lines 193 to 195

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Strip markers that follow preserved bracket prefixes

When remove_subject_markers is enabled, subjects that begin with analyst metadata followed by a reply marker now keep the marker because the anchored regex only matches after whitespace. For example, [CASE-123] RE: malware returns [CASE-123] RE: malware instead of preserving the prefix while stripping the marker to [CASE-123] malware, so common ticket/TLP-prefixed replies are no longer normalized by this option. Consider matching the bracketed prefix separately and retaining it while removing the following marker.

Useful? React with 👍 / 👎.

(?:\[\d+\])?
Expand Down
31 changes: 31 additions & 0 deletions tests/test_email_import_subject_markers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import sys
from types import ModuleType


def load_remove_common_subject_markers():
pymisp = ModuleType("pymisp")
tools = ModuleType("pymisp.tools")
tools.EMailObject = object
tools.URLObject = object
tools.make_binary_objects = lambda *args, **kwargs: None
sys.modules.setdefault("pymisp", pymisp)
sys.modules.setdefault("pymisp.tools", tools)

from misp_modules.modules.import_mod.email_import import remove_common_subject_markers

return remove_common_subject_markers


def test_remove_common_subject_markers_preserves_meaningful_bracketed_prefixes():
remove_common_subject_markers = load_remove_common_subject_markers()

assert remove_common_subject_markers("[CASE-123] RE: malware") == "[CASE-123] RE: malware"
assert remove_common_subject_markers("[TLP:AMBER] RE: incident") == "[TLP:AMBER] RE: incident"


def test_remove_common_subject_markers_strips_leading_reply_forward_markers():
remove_common_subject_markers = load_remove_common_subject_markers()

assert remove_common_subject_markers("RE: malware") == "malware"
assert remove_common_subject_markers(" Fwd: RE: malware") == "malware"
assert remove_common_subject_markers("AW[2]: WG: incident") == "incident"
Loading