Skip to content

feat: Add toggle option in refine dialog to disable cleaning #2754

@baderdean

Description

@baderdean

Description

Currently, when users run a mining operation in Leadminer, the backend automatically triggers cleaning workers that perform:

  • Email verification
  • Disposable email detection

Users should have the option to disable this cleaning step via a toggle in the refine dialog.

Note: The SQL refine_persons function (which copies from persons to refinedpersons) should keep its cleaning logic (tag filtering, name aggregation). This issue is about the worker-based cleaning (email verification & disposable detection), not the SQL refine function.

Requirements

  1. Frontend: Add a toggle option in the refine dialog to disable cleaning

    • Default state: cleaning enabled (true)
    • Toggle label: "Disable cleaning" or similar
    • No persistency needed (just a session toggle)
    • When disabled, the mining process should skip email verification & disposable detection
  2. Backend: Modify the mining flow to accept a parameter to skip cleaning

    • Pass the toggle value to the mining controller/worker
    • When toggled OFF, skip the cleaning worker tasks

Implementation Notes

  • The refine SQL function (private.refine_persons) should NOT be modified - it already handles cleaning correctly
  • The toggle is for the worker-based cleaning (email verification, disposable detection)
  • No persistency required - it's just an on/off toggle for the current session

Files to Modify

  1. frontend/src/components/mining/table/MiningTable.vue - Add toggle in UI where refine dialog appears
  2. frontend/src/stores/leadminer.ts - Pass the toggle value to mining/cleaning
  3. backend/src/controllers/mining.controller.ts - Accept parameter to control cleaning
  4. Any worker-related files that trigger cleaning based on mining parameters

Acceptance Criteria

  • Toggle appears in the refine dialog/UI
  • Toggle is enabled by default (cleaning ON)
  • When toggle is OFF, mining skips email verification & disposable detection
  • No persistency - just a session toggle

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions