Dev

applications/Unity.GrantManager/documentation/reporting/reporting-configuration.md

@@ -0,0 +1,103 @@
+# Reporting Configuration
+
+## Overview
+
+The Reporting Configuration system allows administrators to define how source field data maps to PostgreSQL database columns for generated reporting views. It supports three correlation providers, each sourcing field metadata from a different system:
+
+| Provider | Source | Correlation ID | Description |
+|----------|--------|----------------|-------------|
+| `formversion` | CHEFS form submissions | Form Version ID | Immutable form schema fields |
+| `worksheet` | Unity.Flex worksheets | Form Version ID | Dynamic worksheet fields |
+| `scoresheet` | Unity.Flex scoresheets | Form ID | Evaluation/scoring fields |
+
+## Architecture
+
+### Layered Components
+
+- **`ReportMappingUtils`** (`Unity.Reporting.Application`) — Static utility methods for column name sanitization, validation, uniqueness enforcement, and mapping creation/update logic.
+- **`ReportMappingService`** (`Unity.Reporting.Application`) — Application service orchestrating CRUD operations, field metadata retrieval, view generation, and provider resolution.
+- **`IFieldsProvider`** (`Unity.Reporting.Application`) — Interface for correlation-specific field metadata extraction and change detection.
+- **`ReportingConfigurationController`** (`Unity.Reporting.Web`) — MVC controller providing AJAX API endpoints for the UI.
+- **`Default.js`** (`Unity.Reporting.Web`) — Client-side DataTable configuration, validation, and provider-aware UI logic.
+
+### Field Providers
+
+Each provider implements `IFieldsProvider`:
+
+- **`FormVersionFieldsProvider`** — Retrieves field metadata from immutable CHEFS form version schemas. Change detection always returns null since form versions are immutable.
+- **`WorksheetFieldsProvider`** — Retrieves field metadata from Unity.Flex worksheet definitions. Supports change detection for dynamic schema evolution.
+- **`ScoresheetFieldsProvider`** — Retrieves field metadata from Unity.Flex scoresheet definitions. Supports change detection for scoring structure changes.
+
+## Default Column Name Generation
+
+When a user creates or saves a report configuration for the first time (or when new fields are discovered during an update), the system auto-generates default column names for fields that don't have user-specified names.
+
+### Provider-Specific Column Name Source
+
+The source used for generating default column names differs by provider:
+
+| Provider | Default Column Name Source | Rationale |
+|----------|---------------------------|-----------|
+| `formversion` | **Key** (CHEFS Property Name, e.g., `firstName`) | CHEFS property names are stable, developer-defined identifiers that produce clean, predictable column names (e.g., `firstname`). Labels can be verbose or contain special characters. |
+| `worksheet` | **Label** (e.g., `First Name`) | Worksheet labels are human-readable names configured by administrators, producing descriptive column names (e.g., `first_name`). |
+| `scoresheet` | **Label** (e.g., `Score One`) | Scoresheet labels provide meaningful, user-facing descriptions that map well to reporting column names. |
+
+### Column Name Sanitization
+
+All auto-generated column names go through the same sanitization pipeline regardless of provider:
+
+1. Convert to lowercase
+2. Replace spaces and hyphens with underscores
+3. Remove all non-alphanumeric characters (except underscores)
+4. Remove consecutive underscores
+5. Trim leading/trailing underscores
+6. Prefix with `col_` if name starts with a digit
+7. Truncate to 60 characters maximum
+8. Ensure uniqueness by appending numeric suffixes (`_1`, `_2`, etc.) when collisions occur
+
+### Implementation Details
+
+The column name source selection is implemented identically on server and client with no cross-field fallbacks:
+
+- **Server-side** — `ReportMappingUtils.GetDefaultColumnNameSource()` returns `field.Key ?? ""` for `formversion`, or `field.Label ?? ""` for all other providers. Used by `CreateNewMap()` and `UpdateExistingMap()` when auto-generating column names for unmapped or newly discovered fields. The provider comparison is case-insensitive via `StringComparison.OrdinalIgnoreCase`.
+- **Client-side** — `getDefaultColumnNameSource()` in `Default.js` uses the same logic: `field.key || ''` when `currentProvider === 'formversion'`, otherwise `field.label || ''`. This is used by `transformFieldsMetadata()` to set initial column name values in the DataTable when no saved configuration exists.
+
+> **Important:** Neither implementation falls back from Key to Label or vice versa. If the source value is null/empty, an empty string is used, and the downstream sanitization produces `"col_1"` as a placeholder. This ensures client and server always produce identical defaults.
+
+### Impact on Existing Configurations
+
+- **Existing saved configurations are not affected.** Column names are persisted in the database; the provider-specific logic only determines defaults for new/unsaved fields.
+- **Existing column names are preserved during updates.** The three-tier priority system (user-provided → existing → auto-generated) ensures established mappings are maintained.
+
+## Column Name Validation
+
+Both client-side and server-side enforce PostgreSQL column name rules:
+
+- Maximum 60 characters
+- Must start with a letter or underscore
+- Contains only letters, digits, and underscores
+- Cannot be a PostgreSQL reserved word
+- Must be unique within the configuration
+
+## View Generation
+
+After saving a configuration, users can generate a PostgreSQL database view:
+
+1. User provides a view name (validated for availability and PostgreSQL compliance)
+2. A background job is queued to create/update the view in the `Reporting` schema
+3. View status is tracked and displayed via a widget
+
+View names follow similar sanitization rules with a 63-character maximum (PostgreSQL identifier limit).
+
+## Configuration Lifecycle
+
+```
+Fields Metadata → [Save] → Configuration → [Generate View] → Database View
+                                ↓
+                          [Delete] → Removes configuration and optionally the view
+```
+
+1. **Initial Load**: Field metadata is fetched from the appropriate provider; default column names are generated.
+2. **Save**: Creates or updates the mapping configuration with user-specified and auto-generated column names.
+3. **Generate View**: Creates a PostgreSQL view in the `Reporting` schema based on the saved mapping.
+4. **Delete**: Removes the configuration and optionally drops the associated database view.

applications/Unity.GrantManager/modules/Unity.Reporting/src/Unity.Reporting.Application.Contracts/Configuration/UpsertColumnMappingDto.cs

@@ -3,7 +3,9 @@
     /// <summary>
     /// Data transfer object containing user-specified field-to-column mapping overrides for report configuration.
     /// Allows users to customize column names for specific fields instead of relying entirely on auto-generated names.
-    /// Fields not included in this mapping will receive auto-generated column names based on their labels.
+    /// Fields not included in this mapping will receive auto-generated column names based on the correlation provider:
+    /// for form versions, column names are derived from field keys (CHEFS Property Names); for worksheets and scoresheets,
+    /// column names are derived from field labels.
     /// </summary>
     public class UpsertColumnMappingDto
     {

applications/Unity.GrantManager/modules/Unity.Reporting/src/Unity.Reporting.Application.Contracts/Configuration/UpsertReportColumnsMapDto.cs

@@ -24,7 +24,8 @@ public class UpsertReportColumnsMapDto
         /// <summary>
         /// Gets or sets the optional column mapping configuration containing user-specified field-to-column mappings.
         /// When provided, these mappings override auto-generated column names for specific fields.
-        /// Empty mappings will result in fully auto-generated column names based on field labels.
+        /// Empty mappings will result in fully auto-generated column names derived from field keys
+        /// (for form versions) or field labels (for worksheets and scoresheets).
         /// </summary>
         public UpsertColumnMappingDto Mapping { get; set; } = new();
     }

applications/Unity.GrantManager/modules/Unity.Reporting/src/Unity.Reporting.Application/Configuration/ReportMappingUtils.cs

@@ -16,6 +16,25 @@ internal static partial class ReportMappingUtils
     {
         private const int MaxColumnNameLength = 60;
 
+        /// <summary>
+        /// Determines the appropriate source value for auto-generating a default column name based on the correlation provider.
+        /// For the "formversion" provider, the field's Key (CHEFS Property Name) is used because it provides more stable,
+        /// developer-friendly identifiers. For all other providers (worksheets, scoresheets), the field's Label is used
+        /// as it provides more human-readable column names.
+        /// </summary>
+        /// <param name="field">The field metadata containing both Key and Label properties.</param>
+        /// <param name="correlationProvider">The correlation provider identifier (e.g., "formversion", "worksheet", "scoresheet").</param>
+        /// <returns>The Key for formversion providers, or the Label (falling back to empty string) for all other providers.</returns>
+        internal static string GetDefaultColumnNameSource(FieldPathTypeDto field, string correlationProvider)
+        {
+            if (string.Equals(correlationProvider, Providers.FormVersion, StringComparison.OrdinalIgnoreCase))
+            {
+                return field.Key ?? string.Empty;
+            }
+
+            return field.Label ?? string.Empty;
+        }
+
         /// <summary>
         /// Generates sanitized and unique PostgreSQL-compatible column names from a dictionary of field keys and their display labels.
         /// Processes each label through sanitization to remove invalid characters, enforces uniqueness with numeric suffixes,
@@ -269,6 +288,8 @@ private static bool IsValidColumnName(string columnName)
         /// derived from field metadata. Prioritizes user-specified column names where provided, while automatically generating
         /// sanitized and unique column names for unmapped fields. Ensures all column names are PostgreSQL-compliant and unique
         /// across the entire mapping configuration.
+        /// For the "formversion" provider, auto-generated column names are derived from the field Key (CHEFS Property Name)
+        /// rather than the Label, providing more stable and developer-friendly default names.
         /// </summary>
         /// <param name="upsertReportColmnsMapDto">DTO containing correlation information and optional user-provided column mappings organized by field path.</param>
         /// <param name="fieldsMap">Tuple containing an array of field metadata (with keys, labels, types, paths) and additional mapping metadata from the correlation provider.</param>
@@ -299,12 +320,14 @@ internal static ReportColumnsMap CreateNewMap(UpsertReportColumnsMapDto upsertRe
             var usedColumnNames = new HashSet<string>(userProvidedMappings.Values, StringComparer.OrdinalIgnoreCase);
 
             // Generate column names for fields without user-provided mappings
+            // For formversion provider, use Key (CHEFS Property Name) as the source; for others, use Label
             var autoGeneratedColumnNames = new Dictionary<string, string>();
             foreach (var field in fieldsMap.Fields)
             {
                 if (!userProvidedMappings.ContainsKey(field.Path))
                 {
-                    var sanitizedName = SanitizeColumnName(field.Label ?? string.Empty);
+                    var columnNameSource = GetDefaultColumnNameSource(field, upsertReportColmnsMapDto.CorrelationProvider);
+                    var sanitizedName = SanitizeColumnName(columnNameSource);
                     var uniqueName = EnsureUniqueness(sanitizedName, usedColumnNames);
                     autoGeneratedColumnNames[field.Path] = uniqueName;
                     usedColumnNames.Add(uniqueName);
@@ -355,6 +378,8 @@ internal static ReportColumnsMap CreateNewMap(UpsertReportColumnsMapDto upsertRe
         /// and user-provided updates. Implements a three-tier priority system: user-provided column names take precedence,
         /// followed by existing column names from the database, with auto-generated names for new fields. Maintains column
         /// name uniqueness across the entire mapping while preserving established mappings where possible.
+        /// For newly discovered fields in the "formversion" provider, auto-generated column names are derived from the
+        /// field Key (CHEFS Property Name) rather than the Label.
         /// </summary>
         /// <param name="updateReportColumnsMapDto">DTO containing optional user-provided column mappings to update or add, organized by field path.</param>
         /// <param name="existing">The existing ReportColumnsMap entity from the database containing current mapping configuration and correlation details.</param>
@@ -412,9 +437,11 @@ internal static ReportColumnsMap UpdateExistingMap(UpsertReportColumnsMapDto upd
                     columnName = existingRow.ColumnName;
                 }
                 // Priority 3: Auto-generate for new fields
+                // For formversion provider, use Key (CHEFS Property Name) as the source; for others, use Label
                 else
                 {
-                    var sanitizedName = SanitizeColumnName(field.Label ?? string.Empty);
+                    var columnNameSource = GetDefaultColumnNameSource(field, updateReportColumnsMapDto.CorrelationProvider);
+                    var sanitizedName = SanitizeColumnName(columnNameSource);
                     columnName = EnsureUniqueness(sanitizedName, usedColumnNames);
                     usedColumnNames.Add(columnName);
                 }

applications/Unity.GrantManager/modules/Unity.Reporting/src/Unity.Reporting.Web/Views/Shared/Components/ReportingConfiguration/Default.js

@@ -467,6 +467,18 @@ $(function () {
         };
     }
 
+    // Helper function to get the default column name source based on the current provider.
+    // Mirrors the server-side ReportMappingUtils.GetDefaultColumnNameSource() logic exactly:
+    //   - formversion: use Key only (CHEFS Property Name), no fallback to Label
+    //   - worksheet/scoresheet: use Label only, no fallback to Key
+    // No cross-field fallback ensures the client and server produce identical defaults.
+    function getDefaultColumnNameSource(field) {
+        if (currentProvider === 'formversion') {
+            return field.key || '';
+        }
+        return field.label || '';
+    }
+
     // Helper function to transform fields metadata (module-level to reduce nesting)
     function transformFieldsMetadata(fieldsMetadata) {
         const items = fieldsMetadata.fields.map(field => ({
@@ -475,7 +487,7 @@ $(function () {
             type: field.type,
             path: field.path,
             dataPath: field.dataPath,
-            columnName: field.label || field.key,
+            columnName: getDefaultColumnNameSource(field),
             typePath: field.typePath
         }));