Skip to content

v0.6.33

Choose a tag to compare

View all tags
@github-actions github-actions released this 31 Jan 21:03
· 166 commits to main since this release
v0.6.33
0856812
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Release v0.6.33 - FamilySearch Normalization & Architecture Refinement

Released: YYYY-MM-DD

Overview

This release normalizes FamilySearch as a downstream provider (equal to Ancestry, WikiTree, etc.) and establishes SparseTree SQLite as the canonical source of truth. Major architectural cleanup removes all legacy fallback code in favor of migration scripts.

New Features

Ancestry Update Automation Page (v0.6.33)

  • New database-scoped page at /db/:dbId/ancestry-update for bulk Ancestry synchronization
  • BFS traversal of direct-ancestor sparse tree from selected root person
  • Configurable generation depth (1-4 or full tree)
  • Per-person pipeline: check Ancestry link, process free hints, download/scrape provider data
  • Real-time SSE progress with emoji-prefixed execution log
  • Test mode (dry run) for previewing what would happen
  • Adds "Ancestry Update" link to database sidebar
  • API endpoints: GET /api/ancestry-update/status, GET /:dbId/events (SSE), POST /:dbId/cancel, GET /:dbId/validate/:personId

Ancestry Free Hints Automation (v0.6.33)

  • New "Hints" button on Ancestry row in Provider Data table to process free record hints
  • Playwright automation navigates to hints page, reviews each hint, accepts and saves to tree
  • Supports checkbox selection for adding related people from hints
  • Includes consecutive failure safeguard (stops after 3 failures) and remaining hints re-check
  • SSE-based progress events for real-time UI updates
  • API endpoints: POST/GET /api/ancestry-hints/:dbId/:personId, cancel, status

Ancestry Vital Info Upload (v0.6.33)

  • Extended Ancestry upload to support birth date, birth place, death date, death place, and name (not just photos)
  • Upload dialog now shows field-by-field comparison between local SparseTree data and cached Ancestry data
  • Browser automation uses Ancestry's Quick Edit sidebar to update person facts
  • Automatically handles Living/Deceased status toggle when setting death information
  • Pre-selects all differing fields for upload convenience

FamilySearch Normalization (v0.6.5)

  • FamilySearch now treated as equal downstream provider (no special priority)
  • Photos use -familysearch suffix like other providers: {personId}-familysearch.jpg
  • Added linkFamilySearch() and getFamilySearchPhotoPath() to augmentation service
  • ID resolution uses alphabetical order (no FamilySearch priority)

Architecture: Source of Truth Principle (v0.6.33)

  • SparseTree SQLite is now the canonical source of truth
  • Provider cache (data/provider-cache/) is for comparison only, not a fallback data source
  • All data reads come from SQLite; provider cache shows what providers have vs what SparseTree has

Bug Fixes

SQLite Vital Event Row ID Preservation (v0.6.33)

  • Changed vital_event INSERT from INSERT OR REPLACE to ON CONFLICT DO UPDATE
  • Preserves row IDs which are critical for local_override references
  • INSERT OR REPLACE was deleting and re-inserting rows, orphaning override references

Improved Vital Event Date Parsing (v0.6.33)

  • Now uses actual birth.date and death.date objects instead of parsing lifespan string
  • Captures full dates (e.g., "1979-07-31") rather than just years from lifespan ("1979-2020")
  • Added parseYearFromDate() function to handle various date formats including BC notation

Ancestry Update Log Memory Fix (v0.6.33)

  • Limited execution log entries to 500 to prevent browser memory issues on large trees
  • Shows notice when older entries are truncated

FamilySearch Upload Improvements (v0.6.33)

  • Auto-refresh FamilySearch cache after successful upload so UI shows updated data
  • Improved photo comparison to detect already-uploaded photos via symlink matching
  • Fixed Google login button selector for updated FamilySearch login page
  • Improved vitals page editing: handles conclusion dialogs and edit buttons
  • Added autocomplete handling for date/place fields (clicks first suggestion)

Scraper Auto-Login (v0.6.33)

  • Scraper now attempts auto-login via Google OAuth when landing on FamilySearch login page
  • Added handleLoginIfNeeded() function to detect login pages and complete OAuth flow
  • Navigates directly to target URL after successful login

Provider Comparison Not Using Local Overrides (v0.6.33)

  • Fixed comparison service using original database values instead of user-applied overrides
  • Added applyLocalOverrides() to fetch and apply vital_event overrides before comparison
  • FamilySearch/Ancestry now correctly show "match" when user has applied their values

Date Format Differences Showing as Different (v0.6.33)

  • Added date normalization to expand month abbreviations (e.g., "AUG" → "august")
  • "29 AUG 1933" now correctly matches "29 August 1933"
  • Reduces false positives in provider comparison diff counts

Unicode Escape in View Mode Selector (v0.6.33)

  • Fixed checkmark showing as literal \u2713 text instead of ✓ in tree view mode dropdown
  • JSX text content requires actual Unicode characters, not escape sequences

Fan Chart Ancestor Count (v0.6.33)

  • Fixed footer showing incorrect count like "109 of 62 ancestors shown"
  • Now correctly counts only ancestors within the visible generation range

Stale Browser Connection Detection (v0.6.33)

  • Added verifyAndReconnect() to browser service that checks both Playwright state and actual browser process
  • Augmentation service now detects and recovers from stale connections before scraping
  • Fixes silent failures when browser was closed but connection state was still "connected"

Auto-Login on Sync (v0.6.33)

  • FamilySearch sync now attempts automatic login when auth token is missing
  • Uses providerService.ensureAuthenticated() to try Google SSO or stored credentials
  • Eliminates need for manual browser login before sync operations
  • Applies to both refreshPerson() and fetchPersonDisplayName() methods

FamilySearch Scraper Not Extracting Data (v0.6.33)

  • Updated scraper selectors to match current FamilySearch DOM structure
  • Name now extracted using [data-testid="fullName"]
  • Birth/death dates now extracted from [data-testid="conclusionDisplay:BIRTH"] and [data-testid="conclusionDisplay:DEATH"] containers
  • Added handling for collapsed Vitals section (auto-expands before extracting)
  • Photo extraction uses avatar container near [data-testid="update-portrait-button"]
  • Fixed TypeScript __name decorator issue causing page.evaluate to fail in browser context
  • Added detailed debug logging for photo extraction troubleshooting
  • Improved UI feedback: loading toast during scrape, success toast shows extracted fields

"Use" Button Not Updating SparseTree Row (v0.6.33)

  • Fixed clicking "Use" on provider field values not updating the SparseTree row display
  • Root causes fixed:
    • Client: onFieldChanged callback now refetches person data after applying override
    • Server: getPerson() now applies local overrides to returned person data
  • Added applyLocalOverridesToPerson() to database service for consistent override application
  • Birth/death date/place values from providers now properly display after clicking "Use"

"Set as Primary" Photo Not Updating (v0.6.33)

  • Fixed photo not updating after clicking "Set as Primary" on provider photos
  • Root causes fixed:
    • getPhotoPath() now checks for primary photo ({personId}.jpg) before provider-suffixed photos
    • Photo priority in PersonDetail header and SparseTree row now shows primary photo first
    • Added Cache-Control: no-cache headers to all photo "exists" endpoints to prevent stale responses
  • Added FamilySearch-specific photo endpoint: GET /augment/:personId/familysearch-photo
  • Added client API methods: getFsPhotoUrl(), hasFsPhotoProvider()
  • Added photoVersion state for cache-busting photo URLs after changes

HMR Context Resilience (v0.6.33)

  • Fixed "useSidebar must be used within a SidebarProvider" error during HMR transitions
  • useSidebar() and useTheme() hooks now return safe defaults during dev hot reloads
  • Prevents crashes when React context identity changes during development

Improvements

FamilySearch Upload Performance (v0.6.33)

  • Eliminated redundant post-login delays in FamilySearch upload automation
  • After Google OAuth redirect, now checks if already on target page before re-navigating
  • Replaced fixed waitForTimeout delays with element-based waitForSelector for responsiveness
  • Added PM2 timestamps to server logs for better debugging (time: true in ecosystem config)

Page-Level Padding Control (v0.6.33)

  • Moved p-6 padding from Layout to individual pages for flexibility
  • Full-height pages (tree views, PersonDetail, SparseTreePage) control their own layout
  • Standard content pages receive consistent padding via their root element
  • Fixed missing padding on SparseTreePage and PersonDetail
  • Removed double padding from AIProviders page (external component handles its own padding)

Tree Visualization Parity with Ancestry.com (v0.6.33)

  • Added 5 tree view modes matching Ancestry.com functionality
  • Fan Chart (default): Radial pedigree with lineage-colored wedges (paternal=blue/teal, maternal=red/coral)
  • Horizontal Pedigree: Root left, ancestors right with expandable nodes
  • Vertical Family: Ancestors top, root middle with generation labels
  • Columns and Focus: Existing views retained
  • URL-based routing: /tree/:dbId/:personId/:viewMode
  • New shared components: TreeCanvas (D3 zoom/pan), TreeControls, AncestorNode
  • New utilities: lineageColors, treeLayout, arcGenerator
  • Fixed classic view bugs: dynamic centering, ResizeObserver instead of setTimeout

Vertical Family View Improvements (v0.6.33)

  • Refactored layout to use multi-pass collision resolution algorithm
  • Parents properly avoid overlapping when multiple branches are expanded
  • Added horizontal coupling bars between parent pairs (matching Ancestry.com style)
  • Fixed expand button centering on cards
  • Expansion now maintains current zoom level and pans to center on expanded node
  • No longer zooms out and shifts to bottom-right when expanding ancestors
  • Fixed expand/collapse button positioning to overlap card tops consistently
  • Moved spouse connection line from middle of card to name/date area

Avatar Placeholders (v0.6.33)

  • Replaced emoji-based avatar placeholders with proper SVG components throughout tree views
  • New AvatarPlaceholder component renders gender-aware silhouettes (male/female/unknown profiles)
  • Updated PersonCard, FocusNavigatorView, GenerationalColumnsView, PedigreeChartView, and SparseTreePage
  • Softened gender color palette for better visual harmony (less saturated blue/pink)

Legacy Code Removal (v0.6.33)

  • Removed all legacy fallback code from services (no more dual-path lookups)
  • Services updated: familysearch-refresh, multi-platform-comparison, augmentation, scraper, person.routes
  • Architecture policy: use migration scripts instead of maintaining legacy paths

Migration Scripts

Photo Migration

  • scripts/migrate-fs-photos.ts - Renames FamilySearch photos from {personId}.jpg to {personId}-familysearch.jpg

Cache Migration

  • scripts/migrate-legacy-cache.ts - Moves JSON files from data/person/ to data/provider-cache/familysearch/

Technical Details

Source of Truth Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     Layer 3: Local Overrides                    │
│  User edits that take precedence and survive provider re-sync   │
├─────────────────────────────────────────────────────────────────┤
│                    Layer 2: Normalized Data (SQLite)            │
│  CANONICAL SOURCE OF TRUTH - all reads come from here           │
├─────────────────────────────────────────────────────────────────┤
│                     Layer 1: Raw Provider Cache                 │
│  For comparison only - shows what providers have                │
└─────────────────────────────────────────────────────────────────┘

Key Principles

  1. Initial Seeding: When indexing, data writes to both SQLite (as record) AND provider-cache (for comparison)
  2. No Fallback Code: Migration scripts upgrade data in place; no dual-path lookups
  3. Explicit Apply: Downloaded provider data is cached but never auto-applied; users click "Use" buttons
  4. Provider Equality: All providers treated equally as downstream data sources

Full Changelog

Full Diff: v0.5.4...v0.6.33

Assets 2
Loading