Unable to identify root cause of conflict between cluster-level and instance-level Performance Insights in RDS Aurora (Terraform)

[jhosmanlizarazo]

Description

When deploying an RDS Aurora PostgreSQL cluster with the terraform-aws-modules/rds-aurora module (~> 10.0), a recurring error occurs when trying to create cluster instances:
KMSKeyNotAccessibleFault: The specified performanceInsightsKMSKeyId arn:aws:kms:us-east-1:xxx doesn't exist, isn't enabled, or isn't accessible with your permissions.

Research Conducted

• Attempted to set cluster_performance_insights_kms_key_id = null to use AWS-managed KMS
• Attempted to set performance_insights_kms_key_id = null on each instance
• Attempted to manually modify Terraform state in S3
• Deleted and recreated the cluster multiple times
• Researched documentation of the terraform-aws-modules/rds-aurora module
• Spent ~4 hours troubleshooting without success

How Gemini Identified the Root Cause

An alternative AI (Google Gemini) analyzed the problem differently and found:

1. Ran grep -rn "performance_insights_enabled" .terraform/modules/rds to inspect the module internals
2. Identified that AWS introduced DB Cluster-level Performance Insights for Aurora, causing the AWS API to complain about conflicting cluster and instance settings when setting KMS keys
3. Found that the module attempts to inherit performance_insights_kms_key_id from cluster to instances
4. Key insight: For Aurora, when Performance Insights is enabled at cluster level, all instances must inherit it. Sending a custom KMS key at instance level is not allowed

Root Cause

AWS has a limitation for Aurora: When Performance Insights is enabled at the cluster level (cluster_performance_insights_enabled = true), all instances must inherit it. Sending a custom performance_insights_kms_key_id at the instance level is not allowed.
The conflict occurs because:

1. The rds-aurora module tries to inherit performance_insights_kms_key_id from cluster to instances
2. If the cluster was previously created with a KMS key (even if instance creation failed), AWS remembers that configuration
3. Attempting to use a different KMS or null triggers the error

Solution

Temporarily disable Performance Insights on the cluster:

module "rds" {
  # ... other parameters ...
  
  cluster_performance_insights_enabled    = false
  cluster_performance_insights_kms_key_id = null
}
After successfully creating the cluster, Performance Insights can be enabled in a subsequent apply if desired.
Impact
- Wasted time: ~4 hours of research
- Required assistance from another AI (Gemini) to identify the root cause in ~15 minutes
Recommendation
1. Improve module documentation to clarify this AWS limitation
2. Consider adding validations in the module to detect this conflict
3. Update examples to include this consideration
Versions
- Terraform: 1.14.7
- AWS Provider: ~> 5.0
- RDS Aurora Module: ~> 10.0

### Plugins

No plugins installed - Using OpenCode CLI directly

### OpenCode version

Latest version (opencode/minimax-m2.5-free)

### Steps to reproduce

1. Create an RDS Aurora PostgreSQL cluster using terraform-aws-modules/rds-aurora (~> 10.0)
2. Enable Performance Insights with a custom KMS key: cluster_performance_insights_enabled = true
3. Attempt to create cluster instances
4. Observe error: KMSKeyNotAccessibleFault - The specified performanceInsightsKMSKeyId doesn't exist

### Screenshot and/or share link

_No response_

### Operating System

macOS (Darwin)

### Terminal

macOS Terminal / iTerm2