Preserve bracketed prefixes when stripping email subject markers#792
Preserve bracketed prefixes when stripping email subject markers#792adulau wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe86bb2fe2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ^\s* | ||
| (?:(?:\[[^\]]+\]\s*)*) | ||
| (?: | ||
| (?:re|fw|fwd|wg|aw|sv|tr|rv|vs) |
There was a problem hiding this comment.
Strip markers that follow preserved bracket prefixes
When remove_subject_markers is enabled, subjects that begin with analyst metadata followed by a reply marker now keep the marker because the anchored regex only matches after whitespace. For example, [CASE-123] RE: malware returns [CASE-123] RE: malware instead of preserving the prefix while stripping the marker to [CASE-123] malware, so common ticket/TLP-prefixed replies are no longer normalized by this option. Consider matching the bracketed prefix separately and retaining it while removing the following marker.
Useful? React with 👍 / 👎.
Motivation
[CASE-123],[TLP:AMBER]) when a reply/forward marker followed, losing analyst-relevant context.RE:,Fwd:and indexed variants) while preserving meaningful bracketed metadata.Description
misp_modules/modules/import_mod/email_import.pyby removing the leading(?:\[[^\]]+\]\s*)*subpattern so the regex only matches the actual reply/forward marker sequence (SUBJECT_MARKER_RE).RE,FW,AW[2]) and repeated stripping viaremove_common_subject_markersunchanged.tests/test_email_import_subject_markers.pythat mockpymisp.toolsto importremove_common_subject_markersand assert preservation of bracketed prefixes and correct stripping of reply/forward markers.Testing
pytest tests/test_email_import_subject_markers.py -q, which executed the added tests and reported2 passed.Codex Task