Feature/skill improvements

.claude/skills/add-cms-connector/SKILL.md

@@ -134,6 +134,20 @@ Before inspecting the sample, fetch the CMS's official docs:
   | url / link | `link` or `url` |
   | taxonomy / category / tag | `taxonomy` |
 
+⚠️ **Complex field gate — run before generating any code:**
+After building the mapping table, scan every row whose sample value is a structured object or array (not a string, number, boolean, or `null`). These are **complex fields**. For each one, **stop and invoke `resolve-complex-field`** before proceeding. Pass: the CMS name, the source field type name, the raw sample value, and the connector file paths (to be created in Steps 2–4). Resume building the connector only after every complex field has been resolved and its converter/schema fix confirmed.
+
+Fields that ALWAYS require `resolve-complex-field`:
+- Rich text / structured text (any RTE format)
+- Portable text / DAST / Slate / block content
+- Modular blocks / page builder sections
+- Nested objects or arrays that map to `group` or `modular_blocks`
+- Anything in the mapping table row for `json` unless you are certain the value is already a CS RTE doc
+
+Fields that do NOT require it (primitive or handled by existing infrastructure):
+- `single_line_text`, `multi_line_text`, `number`, `boolean`, `isodate` — no converter needed
+- `file` / `reference` — handled by `getAllAssets` + reference resolution in `reference/entry-creation.md`
+
 ### Step 2 — Scaffold Layer A: the parser package `upload-api/migration-<cms>/`
 Create the package from `templates/upload-api-package/`. Copy each template file, replacing `<cms>`/`<Cms>`/`<CMS>` placeholders:
 - `package.json`, `tsconfig.json`, `config/index.json`

.claude/skills/add-cms-connector/reference/touchpoints.md

@@ -15,7 +15,7 @@ Line numbers are anchors at time of writing — they drift. Always open the file
 - `interface/interface.ts` — `Field`, `FieldAdvanced`, `DataConfig`, `CT = Field[]`. Keep `Field` identical to wordpress.
 - `config/index.json` — `data: "./cmsMigrationData"`, module dir names (`content_types`, `entries`, `assets`, ...).
 - `libs/extractLocale.ts` — returns `string[]` of locale codes.
-- `libs/contentTypes.ts` — `extractContentTypes(affix, filePath, DataConfig)`; reads sample, groups records by type, writes `content_types/*.json`, returns the parsed CTs.
+- `libs/contentTypes.ts` — `extractContentTypes(affix, filePath, DataConfig)`; reads sample, groups records by type, writes `content_types/*.json`, returns the parsed CTs. Always call `ensureMandatoryFields(schema)` before writing each CT — it must inject both `title` (mandatory) **and** `url` (non-mandatory text field) unconditionally, regardless of `options.is_page`. Contentstack rejects any CT missing a `url` field when the CT is page-type, and `is_page` defaults to `true` for most models — do not rely on the flag.
 - `libs/schemaMapper.ts` — switch: source field/widget type → `Field` with chosen `contentstackFieldType`.
 - `utils/helper.ts` — file I/O helpers.
 

.claude/skills/add-cms-connector/templates/api-service.ts

@@ -113,12 +113,25 @@ function transformField(
       return typeof value === 'string' ? value : String(value ?? '');
 
     case 'html':
-    case 'json':
-      // RTE: convert your source rich text into a Contentstack JSON-RTE doc
-      // ({ type:'doc', uid, attrs:{}, children:[{type:'p',uid,attrs:{},children:[{text}]}] }).
-      // Don't drop media embedded IN the rich text — emit embedded-asset nodes
-      // (resolve via assetLookup; shape in reference/entry-creation.md).
-      return value;
+      return typeof value === 'string' ? value : '';
+
+    case 'json': {
+      // Step 1: already a valid CS JSON-RTE doc — pass through untouched.
+      // A valid doc has: { type: 'doc', children: [...] }
+      if (value && typeof value === 'object' && !Array.isArray(value)
+          && (value as any).type === 'doc' && Array.isArray((value as any).children)) {
+        return value;
+      }
+      // Step 2: ADAPT — call your CMS-specific RTE converter here before returning.
+      // e.g. for DatoCMS DAST:      return convertDastToCSRte(value, entryIdMap, recordToCtUid, locale);
+      // e.g. for Portable Text:     return convertPortableTextToCSRte(value, assetLookup);
+      // e.g. for Contentful RT:     return convertContentfulRteToCSRte(value);
+
+      // Step 3: safe fallback — return an empty doc so the entry is never silently dropped.
+      // ⚠️ The CS CLI calls jsonRteData.children.forEach() with no null guard; any non-CS-RTE
+      // value (null, string, DAST object, etc.) crashes and silently drops the WHOLE entry.
+      return { type: 'doc', uid: newUid(), attrs: {}, children: [{ type: 'p', uid: newUid(), attrs: {}, children: [{ text: '' }] }] };
+    }
 
     case 'isodate': {
       if (!value) return null;
@@ -227,6 +240,10 @@ async function createEntry(
   master_locale: string,
   _project: any,
 ): Promise<void> {
+  // ⚠️ Every fs.promises call in this function MUST be awaited. A missing await
+  // (fire-and-forget) means the function returns before writes complete — the CS CLI
+  // then reads files that don't exist yet, producing only partial output (e.g. only
+  // 1 entry visible) with NO error shown.
   try {
     const locale = master_locale || 'en-us';
 
@@ -292,6 +309,9 @@ async function createEntry(
 
       const folderPath = path.join(DATA, destinationStackId, ENTRIES_DIR_NAME, folderName, locale);
       await fs.promises.mkdir(folderPath, { recursive: true });
+      // ⚠️ BOTH files are required. CS CLI reads index.json first as a manifest — if it is
+      // missing, indexerCount = 0 and the entry loop never runs for this CT. The actual entry
+      // data in <locale>.json is never reached and zero entries are created (no error shown).
       await fs.promises.writeFile(path.join(folderPath, `${locale}.json`), JSON.stringify(entryData, null, 4), 'utf-8');
       await fs.promises.writeFile(path.join(folderPath, 'index.json'), JSON.stringify({ '1': `${locale}.json` }, null, 4), 'utf-8');
       console.info(`[<cms>] ${ct?.contentstackUid}: wrote ${Object.keys(entryData).length} entries`);
@@ -323,6 +343,9 @@ async function getAllAssets(
 ): Promise<void> {
   const assetsSave = path.join(DATA, destinationStackId, ASSETS_DIR_NAME);
   await fs.promises.mkdir(path.join(assetsSave, 'files'), { recursive: true });
+  // ⚠️ BOTH files are required. CS CLI reads assets.json first as a manifest — if it is
+  // missing, indexerCount = 0 and the upload loop never runs. The actual asset records in
+  // index.json are never reached and zero assets are uploaded (no error shown).
   await fs.promises.writeFile(path.join(assetsSave, ASSETS_FILE_NAME), JSON.stringify({ '1': ASSETS_SCHEMA_FILE }, null, 4));
   await fs.promises.writeFile(path.join(assetsSave, ASSETS_FOLDER_FILE_NAME), '{}');
 

.claude/skills/add-connector-field/SKILL.md

@@ -12,6 +12,8 @@ A field flows through two mapping points. To support a new source field type (or
 > Tone: same meme rules as `add-cms-connector` — real-time, chat-narration only,
 > one per message max; never in code, tables, or option labels.
 
+⚠️ **Wrong skill for complex fields.** If the field's sample value is a structured object or array — rich text, DAST, portable text, modular blocks, page sections, nested objects — this skill is NOT the right tool. Use `resolve-complex-field` instead. It performs structural analysis, identifies the node format, generates a converter function, and runs a smoke test. This skill only handles simple 1:1 type remappings (e.g. changing a `string` from `single_line_text` to `multi_line_text`, or mapping a previously-dropped `date` field to `isodate`).
+
 ## Inputs you need from the user
 1. **Which connector** — `wordpress` | `contentful` | `drupal` | `aem` | `sitecore` | `sanity`.
 2. **A sample of the field's value** from a real export (so you can see the source type id and the value shape).