New in Terraform: Handle international secondary index drift in Amazon DynamoDB

Should you’ve ever adjusted Amazon DynamoDB international secondary index capability (GSI) outdoors Terraform, you understand how Terraform detects drift and forces undesirable reverts. With Terraform’s new aws_dynamodb_global_secondary_index useful resource, you possibly can tackle this downside.

The brand new aws_dynamodb_global_secondary_index useful resource treats every GSI as an unbiased useful resource with its personal lifecycle administration. You should use this function to make capability changes for GSI and tables outdoors of Terraform.

On this put up, I display easy methods to use Terraform’s new aws_dynamodb_global_secondary_index useful resource to handle GSI drift selectively. I stroll you thru the restrictions of present approaches and information you thru implementing the answer.

The issue: Terraform drift and GSI administration

Earlier than diving into the answer, let’s set up what drift means in infrastructure administration. In infrastructure as code (IaC), drift happens when the precise state of your infrastructure differs from what’s outlined in your Terraform configuration. Terraform detects drift by evaluating the specified state (your .tf configuration information), the final identified state (saved in terraform.tfstate), and the precise state (queried from AWS). When these don’t match, Terraform reviews drift and proposes adjustments to reconcile the distinction.

DynamoDB GSIs typically require capability changes for varied operational causes: load testing, capability planning, emergency efficiency necessities, or managing heat throughput. Your DynamoDB capability will also be modified by autoscaling occasions. Everytime you make these adjustments outdoors of Terraform, it creates drift between Terraform’s configuration and AWS actuality.

For instance, let’s assume your analytics crew runs a each day report that queries a GSI closely. The report runs at 2:00 AM and desires 50 learn capability models (RCUs), however throughout regular hours, 5 RCUs is adequate. Your operations crew manually will increase capability earlier than the report runs to deal with the load.

At 1:50 AM, your ops crew will increase capability from 5 to 50 utilizing AWS Command Line Interface (AWS CLI). The report runs from 2:00 AM to three:00 AM with the upper capability. Later that day, if you run terraform plan to deploy an unrelated change, Terraform detects drift as a result of the precise capability (50) doesn’t match your configuration (5). Terraform desires to revert the capability again to five, which might intrude along with your operational capability administration.

The widespread workaround and its limitations

A standard workaround is to make use of ignore_changes = [global_secondary_index] in your desk’s lifecycle block. This prevents Terraform from detecting capability drift. Nevertheless, this method is simply too broad—it ignores all GSI adjustments, not simply capability. As a result of global_secondary_index is a fancy nested sort, ignore_changes solely works on the top-level, not at particular person attributes. If somebody unintentionally deletes a GSI or modifies its key schema, Terraform gained’t detect it. You’ll be able to’t distinguish between intentional capability tuning and unintentional GSI deletions.

The answer: Separate GSI sources

The brand new aws_dynamodb_global_secondary_index useful resource treats every GSI as an unbiased useful resource with its personal lifecycle administration. This offers you granular management over which attributes to disregard for every GSI whereas nonetheless detecting vital adjustments like deletions or schema modifications.

Stipulations

Earlier than you start, confirm you have got:

The aws_dynamodb_global_secondary_index useful resource is at the moment marked as experimental within the Terraform AWS supplier. This implies the schema or habits would possibly change with out discover, and isn’t topic to the backwards compatibility assure of the supplier.

You should set the atmosphere variable TF_AWS_EXPERIMENT_dynamodb_global_secondary_index to allow this experimental useful resource. With out this atmosphere variable, Terraform will return an error when trying to make use of aws_dynamodb_global_secondary_index. Set it earlier than working any Terraform instructions:

export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1

Take a look at totally in non-production environments earlier than utilizing in manufacturing. You might be welcome to offer suggestions at GitHub Challenge #45640.

Should you’re upgrading from AWS Supplier v5.x to v6.x, overview the v6.0.0 improve information for breaking adjustments earlier than continuing.

Set up Terraform on Amazon Linux:

# Replace system
sudo yum replace -y

# Set up yum-config-manager
sudo yum set up -y yum-utils

# Add HashiCorp repository
sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo

# Set up Terraform
sudo yum -y set up terraform

# Confirm set up
terraform --version

Utilizing the brand new useful resource

Create a provisioned capability desk with two GSIs utilizing the brand new separate useful resource methodology. You’ll create major.tf the place the desk and GSIs are outlined as unbiased sources.

Desk and GSI keys:

Configuration:

terraform {
  required_providers {
    aws = {
      supply  = "hashicorp/aws"
      model = "~> 6.28"
    }
  }
}

supplier "aws" {
  area = "us-east-1"
}

# DynamoDB Desk with out GSI blocks (GSIs managed individually)
# Solely outline attributes which are used as desk keys (hash_key/range_key)
# GSI attributes are outlined within the separate aws_dynamodb_global_secondary_index sources
useful resource "aws_dynamodb_table" "test_table" {
  identify           = "GSITestTable"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  hash_key       = "id"
  range_key      = "timestamp"

  attribute {
    identify = "id"
    sort = "S"
  }

  attribute {
    identify = "timestamp"
    sort = "N"
  }

  tags = {
    Identify        = "GSITestTable"
    Setting = "check"
    Objective     = "Testing new GSI useful resource"
  }
}

# GSI as a separate useful resource
useful resource "aws_dynamodb_global_secondary_index" "status_index" {
  table_name = aws_dynamodb_table.test_table.identify
  index_name = "StatusUserIndex"

  # Provisioned throughput configuration
  provisioned_throughput {
    read_capacity_units  = 5
    write_capacity_units = 5
  }

  # key_schema now consists of attribute_type (required in new useful resource)
  key_schema {
    attribute_name = "standing"
    attribute_type = "S"
    key_type       = "HASH"
  }

  key_schema {
    attribute_name = "user_id"
    attribute_type = "S"
    key_type       = "RANGE"
  }

  # Projection configuration
  projection {
    projection_type = "ALL"
  }

  # With the brand new separate useful resource, now you can ignore particular attributes per GSI
  lifecycle {
    ignore_changes = [provisioned_throughput]
  }
}

# Second GSI to check a number of unbiased GSIs
useful resource "aws_dynamodb_global_secondary_index" "timestamp_index" {
  table_name = aws_dynamodb_table.test_table.identify
  index_name = "TimestampIndex"

  # Provisioned throughput configuration
  provisioned_throughput {
    read_capacity_units  = 3
    write_capacity_units = 3
  }

  key_schema {
    attribute_name = "timestamp"
    attribute_type = "N"
    key_type       = "HASH"
  }

  # Projection configuration
  projection {
    projection_type = "KEYS_ONLY"
  }

  # This GSI is totally managed by Terraform (no ignore_changes)
}

Deploy the sources:

# Set the required atmosphere variable
export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1

terraform init
terraform plan
terraform apply

Take a look at selective ignore_changes by manually altering the capability of StatusUserIndex (the one with ignore_changes):

aws dynamodb update-table 
  --table-name GSITestTable 
  --region us-east-1 
  --global-secondary-index-updates '[{
    "Update": {
      "IndexName": "StatusUserIndex",
      "ProvisionedThroughput": {
        "ReadCapacityUnits": 10,
        "WriteCapacityUnits": 10
      }
    }
  }]'

# Await replace to finish
sleep 30

Operating terraform plan reveals No adjustments regardless that StatusUserIndex capability modified to 10/10 in AWS. This happens due to ignore_changes = [provisioned_throughput].

Confirm drift detection nonetheless works by manually altering TimestampIndex (the one with out ignore_changes):

aws dynamodb update-table 
  --table-name GSITestTable 
  --region us-east-1 
  --global-secondary-index-updates '[{
    "Update": {
      "IndexName": "TimestampIndex",
      "ProvisionedThroughput": {
        "ReadCapacityUnits": 8,
        "WriteCapacityUnits": 8
      }
    }
  }]'

# Await replace to finish
sleep 30

Operating terraform plan detects the drift and proposes to alter TimestampIndex capability from 8 again to three. This demonstrates that:

• StatusUserIndex reveals no adjustments (capability ignored as meant)
• TimestampIndex reveals drift detection (capability adjustments detected)
• Every GSI has unbiased lifecycle administration
• You’ll be able to selectively ignore particular attributes per GSI
• Terraform nonetheless detects vital adjustments on GSIs with out ignore_changes.

The important thing variations from the normal methodology are that the desk defines attributes utilized by the desk itself (id, timestamp), whereas GSI-specific attributes (standing, user_id) are outlined within the separate GSI useful resource’s key_schema blocks with their attribute_type (required in new useful resource). If a GSI reuses a desk attribute, that attribute stays within the desk’s attribute block. The GSI is a separate useful resource with its personal lifecycle.

Advantages of the brand new useful resource

The brand new useful resource mannequin supplies a number of benefits. Now you can ignore particular attributes of a GSI with out affecting different GSIs and automatic scripts can modify capability primarily based on visitors patterns with out creating Terraform drift. You continue to observe vital adjustments like key schema modifications, confirming no unintentional GSI deletions or reconfigurations. Terraform state stays the supply of reality for GSI construction, whereas DynamoDB APIs present precise runtime capability.

Every GSI can have its personal lifecycle guidelines, offering unbiased administration. The brand new useful resource mannequin follows Terraform finest practices the place every useful resource manages one logical infrastructure part, dependencies are specific by means of useful resource references, and state administration is extra easy.

The brand new useful resource totally helps heat throughput configuration for on-demand tables. Heat throughput is a DynamoDB functionality that you should use to specify baseline capability for on-demand tables, serving to you handle efficiency and prices extra predictably. That is how one can check it.

Create ondemand.tf:

useful resource "aws_dynamodb_table" "ondemand_test" {
  identify         = "OnDemandGSITest"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "id"

  attribute {
    identify = "id"
    sort = "S"
  }
}

useful resource "aws_dynamodb_global_secondary_index" "category_index" {
  table_name = aws_dynamodb_table.ondemand_test.identify
  index_name = "CategoryIndex"

  key_schema {
    attribute_name = "class"
    attribute_type = "S"
    key_type       = "HASH"
  }

  # Projection configuration
  projection {
    projection_type = "ALL"
  }

  # Heat throughput configuration (attribute, not a block)
  warm_throughput = {
    read_units_per_second  = 13000
    write_units_per_second = 5000
  }

  lifecycle {
    # Permit guide heat throughput tuning
    ignore_changes = [warm_throughput]
  }
}

Deploy and check:

# Set the required atmosphere variable if not already set
export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1

terraform apply

# Change heat throughput manually
aws dynamodb update-table 
  --table-name OnDemandGSITest 
  --region us-east-1 
  --global-secondary-index-updates '[{
    "Update": {
      "IndexName": "CategoryIndex",
      "WarmThroughput": {
        "ReadUnitsPerSecond": 14000,
        "WriteUnitsPerSecond": 5100
      }
    }
  }]'

# Await replace
sleep 30

# Run terraform plan
terraform plan

Terraform reveals No adjustments as a result of heat throughput adjustments are ignored as anticipated.

Earlier than transferring to the following part, destroy the on-demand check sources:terraform destroy

Migration instance

Now that you simply’ve seen how the brand new useful resource works, let’s stroll by means of a whole hands-on migration of present infrastructure. Begin with a desk utilizing the normal nested GSI method, then migrate it to the brand new separate useful resource methodology with none downtime.

Step 1: Create infrastructure with conventional methodology

Create a DynamoDB desk with a GSI utilizing the normal nested block method.

Create a file referred to as migration-old.tf:

terraform {
  required_providers {
    aws = {
      supply  = "hashicorp/aws"
      model = "~> 6.28"
    }
  }
}

supplier "aws" {
  area = "us-east-1"
}

# Conventional method: GSI outlined as nested block
useful resource "aws_dynamodb_table" "merchandise" {
  identify           = "ProductsTable"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  hash_key       = "ProductId"

  attribute {
    identify = "ProductId"
    sort = "S"
  }

  attribute {
    identify = "Class"
    sort = "S"
  }

  # GSI outlined as nested block (TRADITIONAL METHOD)
  global_secondary_index {
    identify               = "CategoryIndex"
    hash_key           = "Class"
    projection_type    = "ALL"
    read_capacity      = 3
    write_capacity     = 3
  }

  tags = {
    Identify        = "ProductsTable"
    Setting = "migration-demo"
  }
}

Deploy this infrastructure:

terraform init
terraform plan
terraform apply

Confirm the desk and GSI had been created:

aws dynamodb describe-table --table-name ProductsTable --region us-east-1 
  --query 'Desk.GlobalSecondaryIndexes[0].IndexName'

Output:

Step 2: Put together for migration

Earlier than migrating, backup your Terraform state:

terraform state pull > backup-before-migration.tfstate

Set the required atmosphere variable:

export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1

Step 3: Replace your Terraform configuration

Create a brand new file referred to as migration-new.tf with the up to date configuration. Maintain each information for now—you’ll take away the outdated one after import.

terraform {
  required_providers {
    aws = {
      supply  = "hashicorp/aws"
      model = "~> 6.28"
    }
  }
}

supplier "aws" {
  area = "us-east-1"
}

# Up to date desk: GSI block eliminated
useful resource "aws_dynamodb_table" "merchandise" {
  identify           = "ProductsTable"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  hash_key       = "ProductId"

  # Solely outline attributes utilized by the desk's personal keys
  attribute {
    identify = "ProductId"
    sort = "S"
  }

  tags = {
    Identify        = "ProductsTable"
    Setting = "migration-demo"
  }
}

# NEW: GSI as a separate useful resource
useful resource "aws_dynamodb_global_secondary_index" "category_index" {
  table_name = aws_dynamodb_table.merchandise.identify
  index_name = "CategoryIndex"

  # Provisioned throughput configuration
  provisioned_throughput {
    read_capacity_units  = 3
    write_capacity_units = 3
  }

  key_schema {
    attribute_name = "Class"
    attribute_type = "S"
    key_type       = "HASH"
  }

  # Projection configuration
  projection {
    projection_type = "ALL"
  }

  # Permit ops crew to regulate capability with out Terraform reverting it
  lifecycle {
    ignore_changes = [provisioned_throughput]
  }
}

Step 4: Take away the outdated configuration

Now take away or rename the outdated file:

mv migration-old.tf migration-old.tf.backup

At this level, in the event you run terraform plan, you’ll see that Terraform desires to take away the GSI from the desk (as a result of the nested block is gone) and create a brand new separate GSI useful resource.

Don’t apply but. This is able to trigger downtime. As an alternative, import the prevailing GSI.

Step 5: Import the prevailing GSI

Import the prevailing GSI into the brand new useful resource’s state:

# Import format: 'table_name,index_name'
terraform import aws_dynamodb_global_secondary_index.category_index 
  'ProductsTable,CategoryIndex'

Output:

aws_dynamodb_global_secondary_index.category_index: Importing from ID "ProductsTable,CategoryIndex"...
aws_dynamodb_global_secondary_index.category_index: Import ready!
  Ready aws_dynamodb_global_secondary_index for import
aws_dynamodb_global_secondary_index.category_index: Refreshing state... [id=ProductsTable,CategoryIndex]

Import profitable!

Step 6: Confirm the migration

Run terraform plan to confirm:

Anticipated output:

aws_dynamodb_table.merchandise: Refreshing state... [id=ProductsTable]
aws_dynamodb_global_secondary_index.category_index: Refreshing state... [id=ProductsTable,CategoryIndex]

No adjustments. Your infrastructure matches the configuration.

Should you see No adjustments, the migration was profitable. The GSI is now managed as a separate useful resource.

Migration abstract

To finish a migration, you began with a conventional nested GSI configuration, which you then migrated to separate GSI sources with out downtime utilizing terraform import. You then verified the migration with terraform plan displaying No adjustments, after which you efficiently transitioned to the brand new useful resource mannequin.

Key takeaways:

• Migration makes use of terraform import
• No AWS sources are modified or recreated
• The GSI continues to exist all through the migration with zero downtime
• After migration, you have got granular management over what to disregard with ignore_changes
• The migration course of is protected and reversible

Migration concerns

Don’t mix aws_dynamodb_global_secondary_index sources with global_secondary_index blocks on aws_dynamodb_table. Doing so would possibly trigger conflicts, perpetual variations, and GSIs being overwritten.

When migrating, observe these steps:

1. Backup state: terraform state pull > backup.tfstate
2. Set atmosphere variable: export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1
3. Replace configuration: Take away the GSI block from desk the desk and create a brand new GSI useful resource
4. Import present GSI: terraform import 'table_name,index_name'
5. Confirm: Run terraform plan, it ought to present No adjustments
6. Take a look at: Manually change capability and confirm that Terraform ignores the change

You gained’t expertise downtime throughout migration if completed appropriately utilizing terraform import. The GSI continues to exist in AWS all through the migration. The terraform import command solely updates Terraform’s state file—it doesn’t modify AWS sources.

In case your desk has a number of GSIs, migrate them one after the other:

1. Import the primary GSI and confirm with terraform plan
2. Import the second GSI and confirm with terraform plan
3. Proceed till all GSIs are migrated

This reduces threat and simplifies troubleshooting.

Comparability: Conventional in comparison with new methodology

The next desk summarizes the important thing variations between the normal nested block method and the brand new separate useful resource methodology:

Clear up

To keep away from incurring future fees, delete the sources you created on this walkthrough:

# Destroy all Terraform-managed sources
terraform destroy

# Verify the deletion when prompted
# Sort 'sure' to proceed

Should you created any sources manually throughout testing, ensure to delete these as nicely by means of the AWS Administration Console or AWS CLI to keep away from incurring future prices.

Conclusion

On this put up, I confirmed you ways the brand new aws_dynamodb_global_secondary_index useful resource solves the long-standing problem of managing DynamoDB GSI drift in Terraform. The all-or-nothing nature of ignoring nested global_secondary_index blocks created a niche between operational flexibility and infrastructure governance.

By treating GSIs as first-class sources, you achieve granular management with selective ignore_changes for particular GSI attributes, unbiased administration the place every GSI has its personal lifecycle guidelines, higher drift detection that tracks vital adjustments whereas permitting operational changes, and a extra easy structure with separation of considerations between desk and index configuration.

Keep in mind that the aws_dynamodb_global_secondary_index useful resource is at the moment marked as experimental. Whereas it supplies highly effective capabilities for managing GSI drift, bear in mind that:

• The schema or habits would possibly change in future supplier variations
• You should set the atmosphere variable TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1 to allow this useful resource
• It’s not topic to the backwards compatibility assure of the supplier
• You’ll be able to’t combine this useful resource with conventional global_secondary_index blocks on the identical desk

All the time check totally in non-production environments and monitor supplier launch notes for updates. When you’ve got suggestions, present it at GitHub Challenge #45640 to assist form the way forward for this function.

In regards to the authors