fix(converter): robustness + visual fidelity fixes from 140-deck real-world corpus#216
Open
naohito2000 wants to merge 21 commits into
Open
fix(converter): robustness + visual fidelity fixes from 140-deck real-world corpus#216naohito2000 wants to merge 21 commits into
naohito2000 wants to merge 21 commits into
Conversation
added 6 commits
July 24, 2026 17:55
…ting Ran pptx_to_json over 140 real presentations (Canva / Google Slides exports, business templates, AWS decks). Result before: 12 crashes + silent text loss. After: 0 crashes, 0 missing text lines. 1. Chart per-point colors (c:dPt) were stored with int dict keys; minimal-mode _strip_internal_keys calls k.startswith() and crashed on every deck containing such charts (AttributeError). Store str keys (matches builder, which does int(pt_idx_str)) and make the strip tolerant of non-str keys. 2. Duplicate placeholder idx on one slide (Google Slides exports emit two TITLE/idx=0 shapes) — the second shape's text was silently dropped. Rescue extras as positioned textbox elements. 3. Multi-paragraph title placeholders lost every paragraph after the first (only paragraphs[0] was styled). Style and join all paragraphs. Regression tests included; each fails against the pre-fix converter.
Rendered original vs converted-and-rebuilt PPTX to PNG (LibreOffice + pdftoppm) for a 15-deck sample and pixel-diffed each slide: 1. Hidden slides (show="0") were dropped silently: converter now emits "hidden": true, builder restores the attribute. Was the top visual defect — decks with hidden slides rebuilt with page offsets (worst-slide pixel diff 226.9 → 11.5 on a 56-slide deck). 2. Explicit image crop was defeated by fit=contain: with a crop the srcRect fills the frame (PowerPoint semantics), but the builder still shrank the frame to the *uncropped* aspect ratio. Skip contain/cover adjustment when "crop" is present (worst slide 62.0 → 2.6). 3. diff_report never compared "placeholders": hand-edits to titles (the most common edit) were invisible to Workflow C / diff_pptx. Tampered- file test: has_diff was False, now reports placeholder text changes. Regression tests added for all three.
…mode guide Second visual-roundtrip pass over the corpus (140 decks rendered and pixel-diffed): 1. Theme table styles (tableStyleId) resolve to a table-level style dict (body/header/altRow/border). Previously themed tables lost all fills/borders and the builder repainted them with its default banding — white rows over dark decks (worst slide 74.8 → 7.2). 2. Gradient / picture-fill slide backgrounds are no longer dropped: the converter emits a bottom-of-z-order full-slide gradient shape or cover image (builder itself only supports solid backgrounds). Section-divider slides went from white-on-white to faithful (91.7 → 7.2, 80.2 → 2.2, 145.9 → 12.8). 3. Builder: unknown element types now print a warning instead of silently vanishing (this masked bug 2 — a mistyped element was dropped with no trace). 4. import-pptx guide (option b per design discussion): Step 4 now spreads _originalEffects (crop/mask/brightness) into image elements for faithful reproduction; builder contract unchanged. Regression tests: theme table style resolution, gradient-bg extraction, image-bg extraction.
Slides using PowerPoint's 'Hide Background Graphics' lost the flag on conversion; on rebuild the master's decoration (e.g. a white cover shape) reappeared over the slide's own background fill, rendering white-on-white (worst slide pixel diff 99.7 → 1.7). Converter emits "hideMasterShapes": true; builder restores the attribute. Regression test included.
Chased every slide still scoring >50 pixel-diff in the visual roundtrip: 1. SVG artwork recolored to theme color: imported SVGs (waves, decorative graphics) were treated as theme icons and repainted. New iconColor "none" opt-out; converter sets it on extracted SVGs (55.9 → 14.9). 2. SVG-only slide backgrounds (svgBlip extension without raster embed) were skipped (89.3 → 11.6). 3. Images inside raw-XML groups dangled: _groupXml passthrough kept source rIds that mean nothing in the new package. Converter saves a rId→path map (_groupImages); builder re-adds parts and rewrites ids (71.4 → 1.4). 4. Picture-filled shapes (spPr blipFill on textbox/autoshape) dropped the picture; now extracted as a cover image element. 5. WordArt-class shapes (prstTxWarp arch/circle text, per-character picture fill) can't be expressed in the JSON schema — new rawShape element carries the verbatim XML + reattached images (76.8 → 3.9). 6. Run-level text effects (glow/shadow) lost; roundtripped via _textEffects raw XML (part of 53.6 → 4.4). 7. font= styled tag only set a:latin — CJK glyphs kept the theme font; now sets a:ea too (rest of 53.6 → 4.4). 8. lumMod/tint/shade on srgbClr ignored (only schemeClr was handled): new apply_element_transforms; also prstClr preset color names and semi-transparent slide backgrounds (alpha blend) (59.3 → 13.6). 9. Table cells without an anchor rendered middle-aligned: OOXML default is top; converter emits vertical-align explicitly (builder default unchanged — documented schema behavior for AI-authored decks). Not fixed (documented long tail): empty picture-placeholder mosaics (PowerPoint itself renders them blank) and 36%-alpha text (schema has no text opacity). Regression tests for all fix categories.
- Remove unreachable orphaned copy of _extract_svg_blob body left after a past edit inside extract_textbox_element - Consolidate three copies of the 'resolve rId -> save image part' logic (_extract_raw_shape, extract_group_element _groupImages, _extract_blipfill_image) into shared helpers _image_ext / _save_image_part / _save_referenced_images - Output is byte-identical (verified on a 404-slide corpus deck covering rawShape, _groupImages and blipFill paths)
added 15 commits
July 25, 2026 11:06
Connectors carrying xfrm rot= (90/180/270° — common in hand-drawn architecture diagrams) were extracted ignoring the rotation, so rebuilt arrows pointed the wrong way or bent on the wrong axis. - Bake the rotation into the x1/y1/x2/y2 endpoints (the schema has no rotation on lines) - Emit elbowStart="vertical" for 90/270° bent connectors so the builder reconstructs the V-H-V elbow instead of H-V-H Found on a real deck where 11 of 25 connectors on one architecture slide carried rot=. 4 new regression tests.
Two fidelity bugs found on a real architecture deck: - Textboxes with bodyPr anchor= (ctr/b) lost their vertical alignment: the shape path extracted verticalAlign but the textbox path did not, so center-anchored labels rendered top-aligned on rebuild. - <a:sysClr> (Windows system colors, e.g. windowText written by Office when picking 'Automatic' color) resolved to none — visible borders disappeared and fills went transparent. Resolve via lastClr with lumMod/lumOff/tint/shade transforms and alpha applied like srgbClr. 7 new regression tests.
Gradient stop parsing only recognized srgbClr/schemeClr, so stops using sysClr (Office 'Automatic' white) vanished — a green→white 4-stop chevron collapsed to a single green stop and rendered solid. Resolve via the shared _sys_hex helper (lastClr + transforms + alpha) in both fill and line gradient loops. 1 new regression test.
…orms ECMA-376 tint keeps N% of the color (val=100000 = unchanged); the old formula mixed toward white by N% instead, so Office gradient fills with tint/shade/satMod stops (e.g. the classic light-red -> red preset) washed out to white/pale. Also apply satMod (HLS saturation modulation), which was collected from the XML but silently ignored. 3 new regression tests.
…graphs Three fidelity bugs found on a real deck's header/label text: - align="justify" fell back to center in all three shape-builder text paths (the textbox path already supported it) - CJK<->Latin auto-spacing (a typography aid for AI-authored decks) mutated imported text; converted decks now write autoSpacing:false in deck.json and the builder threads the flag down to parse_styled_text - empty paragraphs with an explicit endParaRPr size (spacers that shift center-anchored text) lost their size in the joined-text form; the converter now switches to paragraphs mode with endFontSize 3 new regression tests.
Two label-fidelity bugs found on a real deck: - autoWidth (bodyPr wrap=none) was skipped when the textbox also carried _noAutofit — wrap and autofit are orthogonal, so no-wrap labels re-wrapped mid-word on rebuild - runs with no explicit sz inherit the presentation defaultTextStyle size (OOXML spec fallback: 18pt), but the rebuilt shape got the builder default 14pt; the converter now emits the inherited size explicitly when any run lacks an explicit size 2 new regression tests.
Three table-cell fidelity bugs found on a real deck's spec table: - Cells can carry their own txBody lstStyle whose defRPr sets the text color (white header labels); inheriting runs fell back to the slide tx1 default and rebuilt black. Resolve the cell-level color and emit it as the cell 'color' prop. - sysClr border colors (white gridlines) were dropped — only srgbClr/schemeClr were recognized. Reuse _sys_hex. - buChar bullets inside cells vanished; the builder has no list support in table cells, so keep the bullet character as literal text. 3 new regression tests.
Table cell dicts now accept a paragraphs array:
{"paragraphs": [{"text": "...", "bullet": "\u2022"}, {"text": "..."}]}
Each item becomes its own a:p; bullet items get a real buChar with a
hanging indent, non-bullet items get buNone. Cell-level style props
(color, font-size, weight, align) apply across all paragraphs.
The converter now emits this form for cells with buChar bullets
instead of flattening the bullet character into literal text.
Regression test upgraded to a full convert->build roundtrip.
Cell paragraphs now carry the source marL/indent (EMU) and the builder applies them verbatim, falling back to the generic hanging indent only when unspecified. The hardcoded 171450 EMU was ~2x the real deck's 93663, visibly widening the bullet indent.
python-pptx's add_shape writes a default <p:style> whose effectRef references the theme effect style — a drop shadow in both built-in templates and many real decks. Imported shapes with no effects (empty effectLst, no effectRef) rendered as floating shadowed cards, visibly breaking staircase charts built from tiled white rectangles. Converter now emits _noEffects for shapes whose source defines no effects; the builder drops the default style and writes an empty effectLst (LibreOffice ignores empty effectLst vs effectRef, so the style must go). AI-authored decks are unaffected. 2 new regression tests.
Raw group XML (_groupXml) can contain embedded OLE graphicFrames whose p:oleObj r:id references an oleObject part that does not exist in the rebuilt package. PowerPoint then shows the repair dialog and strips the slide's content (LibreOffice tolerates it, which hid the bug from render-based verification). _reattach_xml_images now sanitizes injected XML first: each OLE graphicFrame is replaced by its mc:Fallback picture, positioned from the frame's p:xfrm; the pic's r:embed is remapped like any other saved image. Verified: rebuilt deck has zero oleObj remnants and no dangling r:id/r:embed/r:link across all slides. 1 new regression test.
Two bugs behind a misdrawn Excel-pasted roadmap chart: - The group child transform only rewrote x/y/width/height, but line elements carry x1/y1/x2/y2 endpoints — connectors inside scaled groups (this deck: scale_y 1.59) kept child-space coordinates and drew far from their true position. Endpoints are now mapped through the group xfrm (top-level and nested groups). - Line solidFill srgbClr ignored lumMod/lumOff transforms (fill side already applied them): light-gray dashed gridlines came back near black. Reuse apply_element_transforms. 1 new regression test.
Vector icon art (here: a cell tower and a network switch) nests groups whose child coordinate space is a few hundred EMU per unit — scaled up ~274x by the parent xfrm. Flattening rounds child coords to px first, collapsing every element to 0x0; the icons disappeared and left stray 12x12 artifacts. Groups with a sub-pixel chExt now fall back to raw XML injection, and the existing freeform-based fallback is applied recursively (icon freeforms live several levels deep). Verified: all 12 corpus-deck slides same or better pixel scores. 2 new regression tests.
Two issues found while round-tripping a 21-deck corpus:
- slide-{NN}.json used fixed 2-digit padding, so slide-100 sorted
before slide-11 for every consumer that orders by filename
(shared/ingest, mcp-local upload_tools, diff). A 352-slide deck
imported with slides 11+ shuffled. Pad width now follows the
slide count (>=2 digits).
- Mirrored pictures dropped flipH/flipV on round-trip: converter
didn't extract them for pictures and the builder didn't apply
them. A flipped cutout photo rendered its subject on the wrong
side (PROFILE deck p4 score 36.9 -> 1.8).
3 new regression tests.
- p:style fontRef text color: shapes styled via theme (white text on accent bars) painted with the deck default. Resolve fontRef schemeClr into elem fontColor when it differs from the default. - Line spacing: a:lnSpc was dropped. Extract spcPct (textbox+shape) and spcPts (new lineSpacingPt, builder support added) — a 48pt title with 31.2pt fixed spacing sat visibly lower without it. - Pattern backgrounds (a:pattFill, e.g. dotGrid graph paper) vanished. Emit a full-slide patternFill rectangle at the bottom of the z-order. - Pictures now round-trip with fit=stretch (OOXML stretch semantics); builder default contain shrank a full-width wave into a blob. - SVG pictures: srcRect crops are baked into the viewBox (crop was silently dropped) and aspect-mismatched frames bake the distortion into the SVG since LibreOffice ignores preserveAspectRatio=none. PROFILE deck mean score 11.5 -> 1.2; no regressions on 3 other decks. 5 new regression tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Hardened
pptx_to_jsonagainst a corpus of 140 real-world PPTX decks (2,097 slides): Google Slides exports, design-heavy marketing templates, WordArt, SVG artwork, tables, charts, videos.Stacked PR — base is
feat/pptx-import-core(#215). Merge order: #215 → #149 → this.Results on the corpus
Fixes (20 bugs, 6 commits)
Crash / data-loss class
_strip_internal_keyshardeneddiff_reportnever compared placeholders — hand-edited titles were invisible to Workflow CFidelity class
show="0") andshowMasterSp="0"now roundtripstyledicticonColor: "none"opt-out of theme recolor); SVG-only blips extracted_groupXmlre-attached via_groupImagesrId map (were dangling refs)rawShaperaw XML_textEffectsa:eatypefacevertical-align: top(OOXML default) instead of inheriting builder's middleRefactor (last commit, byte-identical output)
extract_textbox_element_save_referenced_imageshelpersGuide change (approved approach)
Builder contract unchanged —
_originalEffectsstays opt-in.import-pptxguide Step 4 now includes the spread snippet for faithful reproduction.Testing
tests/test_converter_robustness.py(346 passed total)