Terraform Ephemeral Resources

By coursity / November 27, 2024

Storing sensitive data like database passwords and API keys in Terraform state files poses significant security risks. Terraform 1.10 introduced the ephemeral resource to address this issue, allowing dynamic retrieval of sensitive values during runtime without persisting them in the state file.

Resources

Ephemeral resources are defined using a new block type in HCL:

ephemeral "<resource type>" "<resource name>" { # attributes and nested blocks
}

Enter fullscreen mode
Exit fullscreen mode

These resources are referenced similarly to data sources but are not stored in the state file. Attributes and nested blocks will vary per provider implementation; however, they are referenced similarly to data sources.

Example: Using AWS Secrets Manager with Postgres

In this example, the database credentials are retrieved from AWS Secrets Manager and used to configure the PostgreSQL provider without being stored in the state file.

data "aws_secretsmanager_secret" "example" { name = "myprodsecret"
} ephemeral "aws_secretsmanager_secret_version" "db" { secret_id = data.aws_secretsmanager_secret.example.arn
} locals { credentials = jsondecode(ephemeral.aws_secretsmanager_secret_version.db.secret_string)
} provider "postgresql" { host = "1.2.3.4" port = "5432" username = local.credentials["username"] password = local.credentials["password"]
} data "postgresql_tables" "my_tables" { database = "postgres"
} output "tables" { value = data.postgresql_tables.my_tables.tables.*.object_name
}

Enter fullscreen mode
Exit fullscreen mode

If we examine the items in the state, there is no reference to aws_secretsmanager_secret_version.

# terraform state list
data.aws_secretsmanager_secret.example
data.postgresql_tables.my_tables

Enter fullscreen mode
Exit fullscreen mode

Supported Providers

At the time of writing, the current supported ephemeral blocks are:

Variables and Outputs

Variables can be declared as ephemeral with ephemeral = true.

variable "api_key" { type = string ephemeral = true description = "an api key not stored in state"
}

Enter fullscreen mode
Exit fullscreen mode

Outputs in non-root modules can also be declared as ephemeral:

Example: Using Ephemeral Outputs in Modules

output "database_secret" { ephemeral = true value = aws_secretsmanager_secret_version.db.secret_string
}

Enter fullscreen mode
Exit fullscreen mode

module "database" { source = "./database"
} locals { credentials = jsondecode(module.database.database_secret)
} provider "postgresql" { host = data.aws_db_instance.example.address port = data.aws_db_instance.example.port username = local.credentials["username"] password = local.credentials["password"]
}

Enter fullscreen mode
Exit fullscreen mode

Looking at the state again, there is still no reference to aws_secretsmanager_secret_version:

# terraform state list
data.postgresql_tables.my_tables
module.database.data.aws_secretsmanager_secret.example

Enter fullscreen mode
Exit fullscreen mode

Benefits of Ephemeral Resources

Enhanced Security: Sensitive data is not stored in state files, reducing the risk of unauthorized access.
Dynamic Retrieval: Values are fetched at runtime, ensuring up-to-date information.
Compliance: Helps meet regulatory requirements by keeping secrets out of long-term storage.
Simplified Secret Management: Integrates with secret management solutions like AWS Secrets Manager.

Terraform’s ephemeral feature significantly improves security posture by keeping sensitive data out of state files. By leveraging this feature, teams can maintain the benefits of infrastructure as code while adhering to security best practices. For more detailed information, refer to the Hashicorp documentation on ephemeral resources.

Related

Discover more from Coursity

Subscribe to get the latest posts sent to your email.

Leave a Comment Cancel Reply