Skip to content

feat: add project export and import#1103

Merged
umesh-more-cstk merged 7 commits into
devfrom
feature/exportimportproject
Jun 30, 2026
Merged

feat: add project export and import#1103
umesh-more-cstk merged 7 commits into
devfrom
feature/exportimportproject

Conversation

@yashin4112

Copy link
Copy Markdown
Contributor

🔗 Jira Ticket

. CMG-1005
· CMG-1033


📋 PR Type

  • ✨ Feature
  • 🐛 Bug Fix
  • 🔥 Hotfix
  • ♻️ Refactor
  • 🧹 Chore / Dependency Update
  • 📝 Documentation

📝 Description

What changed?

Project Export / Import (feature)

  • Export: bundles a project's record + its mapper stores (contentTypesMapper, field-mapper, etc.) into a <projectId>.zip, streamed from the backend via archiver. Added an Export Project button in Settings.
  • Import: accepts an exported zip, creates a fresh project under the current user/org with a new id, and restores the mapper stores into database/<newId>/ with their internal projectId references rewritten.
  • New Project is now a dropdown (Create New / Import Existing); the modal is two-step, driven by initialStep (project form vs zip file picker).
  • Backend: exportProject / importProject service + controllers; routes GET /org/:orgId/project/:projectId/export and POST /org/:orgId/project/import (multer upload). Stash/restore token_payload across multer wiping req.body.
  • New deps: archiver, multer, adm-zip.

Sitecore entry title/url (bug fix)

  • Entry title/url were only set inside the field-mapping loop, so entries whose content type had no mappable user fields (only system __ fields) never got a title and were skipped. Now pulled straight from the Sitecore meta block so those entries are created too.

Tests

  • Added export/import coverage for the project service, controller, and routes (vitest).

Why?

  • Lets users move a project (with its content-type/field mappings) between environments or back up/restore it without redoing the mapping work.
  • Fixes Sitecore entries silently dropping when their content type only had system fields.

🧩 Affected Areas

  • api — Node.js backend
  • ui — React frontend
  • upload-api — Upload API server
  • docker / docker-compose
  • CI / GitHub Actions workflows
  • Environment variables / config
  • Other:

🧪 How to Test

Export / Import

  1. Open a project's Settings → click Export Project → a <projectId>.zip downloads (contains project.json + the mapper folder).
  2. On the Projects page, click New ProjectImport Existing → choose the exported zip → Import Project.
  3. Confirm a new project is created (fresh id) under your current org and you land in its migration steps.

Sitecore fix

  1. Run a Sitecore migration where a content type has only system (__) fields.
  2. Confirm entries for that content type now get created with a title/url.

Expected result: Export produces a valid zip; import recreates the project + mappers with a new id; Sitecore entries with no user fields are no longer skipped.


📸 Screenshots / Recordings

Before After

🔗 Related PRs / Dependencies

  • N/A

✅ Author Checklist

  • Branch follows naming convention: feature/, bugfix/, or hotfix/ + 5–30 lowercase chars
  • Jira ticket linked above
  • Self-reviewed the diff — no debug logs, commented-out code, or TODOs left in
  • .env / example.env updated if new environment variables were added
  • No sensitive credentials or secrets committed
  • Existing tests pass locally (npm test)
  • New tests written (export/import service, controller, route coverage)
  • README.md / docs updated if behaviour changed
  • Talisman pre-push scan passes (no secrets flagged)

👀 Reviewer Notes

  • archiver v8 is ESM-only (no factory fn) — we use new ZipArchive(...). archiver/adm-zip/multer are imported untyped, same as the existing express pattern; runs under tsx, flagged only by tsc.
  • Import remaps the original project id everywhere (record + mapper JSON internals) — worth a look at the rewrite logic in importProject.
  • Two unrelated changes ride together (export/import feature + Sitecore fix) — flagging in case you'd prefer them split.

Export bundles a project's record + mapper stores into a <projectId>.zip
(archiver). Import accepts that zip and recreates the project under the current
user/org with a fresh id, rewriting internal projectId references.

- api: export/import service + controllers, routes (multer upload); deps archiver, multer, adm-zip
- ui: Export button in Settings; New Project dropdown (Create New / Import Existing); two-step modal

fix: set Sitecore entry title/url from meta regardless of field mapping

test: add export/import coverage for project service, controller, and routes
@yashin4112 yashin4112 requested a review from a team as a code owner June 29, 2026 05:31
@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 1 0 10 ✅ Passed
🟠 High Severity 14 341 25 ✅ Passed
🟡 Medium Severity 38 0 500 ✅ Passed
🔵 Low Severity 1 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 2 0 90 / 365 days ❌ Failed
🔵 Low 0 0 180 / 365 days ✅ Passed

🟡 Medium Severity - SLA Breached Issues (with fixes)

Showing 2 issue(s) that have exceeded the 90-day SLA threshold:

  1. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@1.9.1
    • Published: 510 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899
  2. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@1.9.1
    • Published: 510 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 341
  • Medium without fixes: 0
  • Low without fixes: 0

❌ BUILD FAILED - Security checks failed

Please review and fix the security vulnerabilities before merging.

@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 1 0 10 ✅ Passed
🟠 High Severity 14 341 25 ✅ Passed
🟡 Medium Severity 38 0 500 ✅ Passed
🔵 Low Severity 1 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 2 0 90 / 365 days ❌ Failed
🔵 Low 0 0 180 / 365 days ✅ Passed

🟡 Medium Severity - SLA Breached Issues (with fixes)

Showing 2 issue(s) that have exceeded the 90-day SLA threshold:

  1. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@2.1.0
    • Published: 510 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899
  2. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@2.1.0
    • Published: 510 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 341
  • Medium without fixes: 0
  • Low without fixes: 0

❌ BUILD FAILED - Security checks failed

Please review and fix the security vulnerabilities before merging.

@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 1 0 10 ✅ Passed
🟠 High Severity 4 337 25 ✅ Passed
🟡 Medium Severity 9 0 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 2 0 90 / 365 days ❌ Failed
🔵 Low 0 0 180 / 365 days ✅ Passed

🟡 Medium Severity - SLA Breached Issues (with fixes)

Showing 2 issue(s) that have exceeded the 90-day SLA threshold:

  1. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@2.1.0
    • Published: 511 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899
  2. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@2.1.0
    • Published: 511 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 337
  • Medium without fixes: 0
  • Low without fixes: 0

❌ BUILD FAILED - Security checks failed

Please review and fix the security vulnerabilities before merging.

@yashin4112 yashin4112 closed this Jun 29, 2026
@yashin4112 yashin4112 reopened this Jun 29, 2026
@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 1 0 10 ✅ Passed
🟠 High Severity 4 337 25 ✅ Passed
🟡 Medium Severity 9 0 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 2 0 90 / 365 days ❌ Failed
🔵 Low 0 0 180 / 365 days ✅ Passed

🟡 Medium Severity - SLA Breached Issues (with fixes)

Showing 2 issue(s) that have exceeded the 90-day SLA threshold:

  1. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@2.1.0
    • Published: 511 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899
  2. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@2.1.0
    • Published: 511 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 337
  • Medium without fixes: 0
  • Low without fixes: 0

❌ BUILD FAILED - Security checks failed

Please review and fix the security vulnerabilities before merging.

@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 1 0 10 ✅ Passed
🟠 High Severity 4 337 25 ✅ Passed
🟡 Medium Severity 9 0 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 2 0 90 / 365 days ❌ Failed
🔵 Low 0 0 180 / 365 days ✅ Passed

🟡 Medium Severity - SLA Breached Issues (with fixes)

Showing 2 issue(s) that have exceeded the 90-day SLA threshold:

  1. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@1.9.1
    • Published: 511 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899
  2. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@1.9.1
    • Published: 511 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 337
  • Medium without fixes: 0
  • Low without fixes: 0

❌ BUILD FAILED - Security checks failed

Please review and fix the security vulnerabilities before merging.

@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 1 0 10 ✅ Passed
🟠 High Severity 4 337 25 ✅ Passed
🟡 Medium Severity 9 0 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 2 0 90 / 365 days ❌ Failed
🔵 Low 0 0 180 / 365 days ✅ Passed

🟡 Medium Severity - SLA Breached Issues (with fixes)

Showing 2 issue(s) that have exceeded the 90-day SLA threshold:

  1. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@2.1.0
    • Published: 511 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899
  2. Regular Expression Denial of Service (ReDoS)

    • ID: SNYK-JS-SHOWDOWN-8685130
    • Package: showdown@2.1.0
    • Published: 511 days ago (SLA: 90 days)
    • CVSS Score: 6.9
    • CVE: CVE-2024-1899

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 337
  • Medium without fixes: 0
  • Low without fixes: 0

❌ BUILD FAILED - Security checks failed

Please review and fix the security vulnerabilities before merging.

@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 0 0 10 ✅ Passed
🟠 High Severity 0 377 25 ✅ Passed
🟡 Medium Severity 0 0 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

✅ No SLA breaches detected. All vulnerabilities are within acceptable time thresholds.

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 0 0 90 / 365 days ✅ Passed
🔵 Low 0 0 180 / 365 days ✅ Passed

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 377
  • Medium without fixes: 0
  • Low without fixes: 0

✅ BUILD PASSED - All security checks passed

@umesh-more-cstk

Copy link
Copy Markdown
Contributor

Reviewed the full diff (service + controller + routes + Sitecore fix + UI service). Solid PR overall — the showdown churn nets out clean (fully removed, no dangling imports). One security blocker before this can merge, rest is minor.

🔴 Blocker — Zip Slip / path traversal in importProject

In projects.service.ts, the destination path is built from the zip entry name without validation:

const relativePath = normalized.slice(oldId.length + 1);
const destPath = path.join(newProjectDir, relativePath);
fs.writeFileSync(destPath, ...);

The zip is fully user-uploaded. A crafted entry like <oldId>/../../../../some/path/x.json will make path.join resolve outside newProjectDir, allowing arbitrary file write/overwrite (other projects' data, config, etc.). Classic Zip Slip. Needs a containment check:

const destPath = path.join(newProjectDir, relativePath);
const resolved = path.resolve(destPath);
const root = path.resolve(newProjectDir);
if (resolved !== root && !resolved.startsWith(root + path.sep)) {
  throw new BadRequestError('Invalid archive: path traversal detected');
}

🟠 Should-fix

  • Untyped imports + tsc. import { ZipArchive } from "archiver" is untyped, so tsc won't catch a wrong import name — a mistake here only surfaces at runtime when the export endpoint is hit. Please confirm the export path was manually tested (zip downloads + opens correctly). Also: does CI run a tsc typecheck step? If so this will fail per the reviewer notes.
  • No rollback on partial import. The project record is pushed to lowdb before the file-copy loop runs. If the loop throws midway (or the zip-slip guard above trips), an orphan project record is left with missing/partial mappers. Suggest pushing the record only after the files are written successfully, or cleaning up on failure.
  • Zip-bomb exposure. adm-zip loads the whole buffer into memory. The multer upload is capped at 100 MB, but the decompressed size is unbounded — a high deflate ratio could blow memory. Worth at least an uncompressed-size check or a note.

🟡 Nits

  • Sitecore fix looks correct — title/url now set from the meta block unconditionally, outside the field-mapping loop, which matches the described bug. Minor: url = /${entry?.meta?.key}`` produces /undefined if `meta.key` is missing — small guard would help.
  • In UI project.service.ts, the catch blocks for exportProject/importProject say Error in userSession: — looks copy-pasted from getProject; the context label is wrong.
  • The token_payload stash via req.tokenPayload works but is fragile (relies on authenticateUser running first). Acceptable, but ideally the auth payload lives outside req.body so multer can't wipe it.

🧹 Commit hygiene

5 of the 6 commits are showdown version thrashing and the PR was closed/reopened once — please squash-merge to keep dev history clean. Also agree with your own note that the export/import feature and the Sitecore fix are unrelated; splitting would be cleaner, but the end state is fine if bandwidth is tight.

- import: Zip Slip guard, zip-bomb size cap, rollback on partial import
- sitecore: skip url when meta.key missing (no /undefined)
- ui: correct export/import error labels
- tests: Zip Slip + zip-bomb coverage
@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 0 0 10 ✅ Passed
🟠 High Severity 0 377 25 ✅ Passed
🟡 Medium Severity 0 0 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

✅ No SLA breaches detected. All vulnerabilities are within acceptable time thresholds.

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 0 0 90 / 365 days ✅ Passed
🔵 Low 0 0 180 / 365 days ✅ Passed

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 377
  • Medium without fixes: 0
  • Low without fixes: 0

✅ BUILD PASSED - All security checks passed

@umesh-more-cstk umesh-more-cstk merged commit 1c07f29 into dev Jun 30, 2026
9 checks passed
@umesh-more-cstk umesh-more-cstk deleted the feature/exportimportproject branch June 30, 2026 05:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants