fix(search): set dynamic:false on all index mappings to stop uncontrolled dynamic mapping#28711
fix(search): set dynamic:false on all index mappings to stop uncontrolled dynamic mapping#28711mohityadav766 wants to merge 2 commits into
Conversation
…lled dynamic mapping
Reproduces and fixes the production reindex failure:
failed to parse field [pipelineStatuses.config.appConfig.actions.customProperties]
of type [text] ... value: '{columndatepivot=jointure}'
pipelineStatuses (and other arbitrary config sub-trees) had no dynamic setting, so
OpenSearch/Elasticsearch dynamically mapped unexpected fields. A string value
auto-typed customProperties as text; a later object value then failed to parse.
Uncontrolled dynamic mapping also causes field-count explosion.
Adds "dynamic": false at the mappings root of every index mapping (all locales).
Unexpected fields are stored in _source but never auto-typed or indexed, so type
conflicts and field explosion cannot happen. All searchable/aggregatable fields
remain explicitly mapped; report-data indices map their data fields explicitly, so
nothing relied on dynamic indexing.
- DynamicMappingTypeConflictTest: standalone OpenSearch testcontainer; reproduces
the type conflict on a dynamic mapping and proves dynamic:false (and the real
ingestion_pipeline mapping) accept both string and object config values.
- IndexMappingNestedFieldConsistencyTest.allIndexMappingsMustDisableDynamic:
guards that every mapping keeps "dynamic": false.
Note: changing mappings triggers a reindex on deploy (mapping version changes).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR hardens OpenMetadata’s Elasticsearch/OpenSearch index mappings by disabling uncontrolled dynamic mapping ("dynamic": false) across all index mapping JSON files to prevent type-conflict reindex failures and field-count explosions from arbitrary config keys.
Changes:
- Set
"mappings": { "dynamic": false, ... }in all shipped index mapping JSONs (all locales + report/analytics indices). - Add a regression test to enforce that all loaded mappings keep
dynamic:false. - Add a Testcontainers-backed repro test that demonstrates the dynamic-mapping type conflict and validates the fix against a real OpenSearch node.
Reviewed changes
Copilot reviewed 243 out of 243 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| openmetadata-spec/src/main/resources/elasticsearch/zh/worksheet_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/user_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/topic_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/test_suite_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/test_case_result_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/test_case_resolution_status_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/test_case_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/team_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/tag_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/table_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/stored_procedure_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/storage_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/spreadsheet_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/security_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/search_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/search_entity_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/query_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/query_cost_record_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/prompt_template_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/pipeline_status_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/pipeline_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/pipeline_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/mlmodel_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/mlmodel_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/metric_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/metadata_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/messaging_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/mcp_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/mcp_server_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/mcp_execution_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/llm_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/llm_model_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/knowledge_page_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/ingestion_pipeline_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/glossary_term_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/glossary_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/folder_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/file_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/drive_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/domain_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/directory_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/database_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/database_schema_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/database_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/data_products_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/dashboard_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/dashboard_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/dashboard_data_model_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/context_memory_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/context_file_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/container_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/column_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/classification_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/chart_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/api_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/api_endpoint_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/api_collection_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/ai_governance_policy_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/zh/ai_agent_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/web_analytic_user_activity_report_data_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/web_analytic_entity_view_report_data_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/raw_cost_analysis_report_data_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/entity_report_data_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/aggregated_cost_analysis_report_data_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/worksheet_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/user_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/topic_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/test_suite_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/test_case_result_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/test_case_resolution_status_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/test_case_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/team_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/tag_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/table_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/stored_procedure_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/storage_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/spreadsheet_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/security_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/search_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/search_entity_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/query_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/query_cost_record_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/prompt_template_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/pipeline_status_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/pipeline_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/pipeline_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/mlmodel_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/mlmodel_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/metric_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/metadata_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/messaging_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/mcp_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/mcp_server_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/mcp_execution_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/llm_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/llm_model_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/knowledge_page_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/ingestion_pipeline_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/glossary_term_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/glossary_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/folder_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/file_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/drive_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/domain_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/directory_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/database_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/database_schema_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/database_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/data_products_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/dashboard_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/dashboard_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/dashboard_data_model_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/context_memory_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/context_file_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/container_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/column_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/classification_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/chart_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/api_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/api_endpoint_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/api_collection_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/ai_governance_policy_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/ru/ai_agent_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/worksheet_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/user_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/topic_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/test_suite_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/test_case_result_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/test_case_resolution_status_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/test_case_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/team_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/tag_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/table_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/stored_procedure_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/storage_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/spreadsheet_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/security_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/search_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/search_entity_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/query_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/query_cost_record_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/prompt_template_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/pipeline_status_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/pipeline_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/pipeline_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/mlmodel_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/mlmodel_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/metric_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/metadata_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/messaging_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/mcp_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/mcp_server_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/mcp_execution_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/llm_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/llm_model_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/knowledge_page_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/ingestion_pipeline_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/glossary_term_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/glossary_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/folder_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/file_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/drive_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/domain_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/directory_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/database_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/database_schema_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/database_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/data_products_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/dashboard_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/dashboard_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/dashboard_data_model_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/context_memory_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/context_file_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/container_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/column_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/classification_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/chart_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/api_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/api_endpoint_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/api_collection_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/ai_governance_policy_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/jp/ai_agent_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/worksheet_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/user_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/topic_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/test_suite_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/test_case_result_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/test_case_resolution_status_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/test_case_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/team_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/tag_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/table_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/stored_procedure_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/storage_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/spreadsheet_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/security_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/search_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/search_entity_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/query_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/query_cost_record_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/prompt_template_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/pipeline_status_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/pipeline_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/pipeline_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/mlmodel_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/mlmodel_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/metric_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/metadata_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/messaging_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/mcp_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/mcp_server_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/mcp_execution_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/llm_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/llm_model_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/knowledge_page_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/ingestion_pipeline_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/glossary_term_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/glossary_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/folder_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/file_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/drive_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/domain_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/directory_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/database_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/database_schema_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/database_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/data_products_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/dashboard_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/dashboard_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/dashboard_data_model_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/context_memory_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/context_file_search_index.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/container_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/column_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/classification_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/chart_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/api_service_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/api_endpoint_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/api_collection_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/ai_governance_policy_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-spec/src/main/resources/elasticsearch/en/ai_agent_index_mapping.json | Disable dynamic mapping at mappings root |
| openmetadata-service/src/test/java/org/openmetadata/service/search/IndexMappingNestedFieldConsistencyTest.java | Add enforcement test for mappings.dynamic=false |
| openmetadata-service/src/test/java/org/openmetadata/service/search/DynamicMappingTypeConflictTest.java | Add real-OpenSearch repro test for dynamic-mapping type conflicts |
| for (Map.Entry<String, JsonNode> entry : allMappings.entrySet()) { | ||
| JsonNode dynamic = entry.getValue().path("mappings").path("dynamic"); | ||
| if (dynamic.isMissingNode() || dynamic.asBoolean(true)) { | ||
| violations.add( | ||
| entry.getKey() + " (dynamic=" + (dynamic.isMissingNode() ? "missing" : dynamic) + ")"); | ||
| } |
| opensearch = | ||
| new GenericContainer<>(DockerImageName.parse("opensearchproject/opensearch:2.13.0")) | ||
| .withEnv("discovery.type", "single-node") | ||
| .withEnv("DISABLE_SECURITY_PLUGIN", "true") | ||
| .withEnv("DISABLE_INSTALL_DEMO_CONFIG", "true") | ||
| .withEnv("OPENSEARCH_JAVA_OPTS", "-Xms1g -Xmx1g") |
| try (InputStream is = | ||
| getClass() | ||
| .getClassLoader() | ||
| .getResourceAsStream("elasticsearch/en/ingestion_pipeline_index_mapping.json")) { | ||
| mapping = new String(is.readAllBytes(), StandardCharsets.UTF_8); | ||
| } |
| HttpRequest request = | ||
| HttpRequest.newBuilder(URI.create(baseUrl + path)) | ||
| .header("Content-Type", "application/json") | ||
| .PUT(HttpRequest.BodyPublishers.ofString(body)) | ||
| .build(); |
…e conflicts
Targeted stopgap for the production reindex failure:
failed to parse field [pipelineStatuses.config.appConfig.actions.customProperties]
of type [text] ... value: '{columndatepivot=jointure}'
pipelineStatuses allowed dynamic mapping of its arbitrary config sub-tree, so a
string value auto-typed customProperties as text and a later object value failed
to parse. Adds "dynamic": false to pipelineStatuses (all locales) so the config
sub-tree is stored in _source but never auto-typed. The comprehensive index-wide
dynamic:false is handled separately (#28711); this gives immediate relief here.
Validated on a real OpenSearch node: the real ingestion_pipeline mapping now
accepts both string and object config values.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🔴 Playwright Results — 38 failure(s), 15 flaky✅ 4226 passed · ❌ 38 failed · 🟡 15 flaky · ⏭️ 89 skipped
Genuine Failures (failed on all attempts)❌
|
|
@mohityadav766 lets file a github issue that can be reproducible. We are not going to merge this. Without clear evidence and testing we are going to add more baggage |
- IndexMappingNestedFieldConsistencyTest: require the boolean false at the mappings root (not asBoolean(true), which let "strict"/non-boolean values pass) and recursively forbid nested "dynamic": true overrides - test_case_resolution_status (en/jp/ru/zh): flip the nested testCaseResolutionStatusDetails "dynamic": true -> false so the root hardening is not re-opened for that subtree (only assignee.name is queried and it stays explicitly mapped, so no search regression) - DynamicMappingTypeConflictTest: pin OpenSearch to 3.4.0 to match the rest of the test/dev infra with a lighter 512m heap, null-check the mapping resource stream, and add per-request + connect HTTP timeouts Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code Review ✅ Approved 2 resolved / 2 findingsHardens index mappings by setting dynamic:false globally to prevent field-count explosions and parsing conflicts. Resolved nested override issues and improved the validation coverage for mapping consistency. ✅ 2 resolved✅ Bug: Nested dynamic:true override defeats hardening in test_case_resolution_status
✅ Quality: Guard test only checks root dynamic, misses nested dynamic:true overrides
OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
|



Problem
Production reindex failure (SPG-DEV and others):
pipelineStatuses(and other arbitrary config sub-trees likeconfig/appConfig) had nodynamicsetting, so ES/OS dynamically mapped unexpected fields. A string value auto-typedcustomPropertiesastext; a later object value ({columndatepivot=jointure}) then failed to parse →mapper_parsing_exception. Uncontrolled dynamic mapping is also the root cause of field-count explosion.This is the "stop dynamic mapping completely" hardening discussed for indexing safety.
Fix
Add
"dynamic": falseat themappingsroot of every index mapping (all locales, 241 files). Unexpected fields are stored in_sourcebut never auto-typed or indexed, so:All searchable/aggregatable fields stay explicitly mapped. Report-data/analytics indices map their
datafields explicitly (verified), so nothing relied on dynamic indexing.dynamic:false(notstrict) is used deliberately —strictwould reject documents carrying config;falseis non-destructive (stores, doesn't reject).Validation
Reproduced and verified against a real OpenSearch node:
DynamicMappingTypeConflictTest— reproduces the exacttextvs object parse failure on a dynamic mapping, and provesdynamic:false(and the realingestion_pipelinemapping) accept both string and object config values.IndexMappingNestedFieldConsistencyTest.allIndexMappingsMustDisableDynamic— guards that every mapping keepsdynamic:false.Existing mapping/search tests pass:
IndexMappingVersionTrackerTest,SearchIndexReindexFieldsParityTest,SearchSourceBuilderFactoryTest,IndexMappingNestedFieldConsistencyTest.Notes
mappings.dynamichere vsmappings.properties.extensionthere).🤖 Generated with Claude Code