☸️ Kubernetes: A Convenient Variable Substitution Mechanism for Kustomize

Kustomize is one of the prominent tools alongside Helm used for managing and configuring Kubernetes deployments across multiple environments. Unlike Helm, which utilizes templates, Kustomize simplifies management by primarily focusing on plain manifests without direct comparison to Helm’s approach.

Variable substitution becomes crucial when environments are ephemeral, as in the case of feature branches, or when configuration settings like DNS entries need to be determined dynamically at deployment time. Kustomize assumes a fixed environment list (e.g., dev/staging/prod), but practical Kubernetes usage often demands more flexibility.

The biggest advantage of Kustomize lies in its straightforward approach to managing manifests. It allows for easy manipulation of fields such as annotations, image names, and replicas across environments. However, Kustomize is strict about variable substitution, preferring a predefined set of options which can be limiting.

To circumvent these limitations, the following official method is often employed within a kustomization.yml file, after creating an environment-specific file:

resources: - deployment.yml - ingress.yml - service.yml - sa.yml - role.yml - role-binding.yml generatorOptions: disableNameSuffixHash: true configMapGenerator:
- name: environment-variables envs: [rendered/api-gateway.env] behavior: create replacements:
- source: kind: ConfigMap version: v1 name: environment-variables fieldPath: data.API_GATEWAY_URL targets: - select: kind: Ingress name: api-gateway fieldPaths: - spec.rules.0.host - spec.tls.0.hosts.0 - select: kind: Deployment name: api-gateway fieldPaths: - spec.template.spec.containers.0.env.[name=API_GATEWAY_URL].value

So much for such a simple need like variable substitution. People will even find Helm appealing at this step. Let’s explain why and how to find a simpler solution.

While Kustomize apply workflow supports basic variable substitution using tools like envsubst on top of kustomize build, more complex scenarios require custom solutions. One more refined approach involves using renvsubst, to selectively replace variables based on specific criteria (e.g., K_* environment-specific or CI_* CI-specific variables).

We could then use in our pipelines, in ascending order of complexity :

• For simple use cases : kustomize build . | envsubst
• To avoid replacing some variables, define it in manifests as ${DOLLAR}MY_VARIABLE and then:
  • kustomize build . | DOLLAR=$ envsubst
• Use renvsubst to choose variables with prefixes:
  • kustomize build . | renvsubst --prefix="K_" --prefix="CI_"

But all these solutions have some drawbacks. You have to pipe the kustomize build result before applying manifests; it breaks some convenient wrapping tools features like intelligent local updates using tilt.dev.

No compromise solution using renvsubst as a transformer

For a no compromise solution using a Kustomize transformer exec plugin using renvsubst, follow these steps.

1. Install renvsubst: Ensure renvsubst is installed from here.

2. set the KUSTOMIZE_PLUGIN_HOME variable for kustomize to find your plugin, for example KUSTOMIZE_PLUGIN_HOME=$PWD/devops/k8s/scripts/

3. Set up Transformer Script:

Create a transformer script (renvsubst) at $KUSTOMIZE_PLUGIN_HOME/transformers/renvsubst/renvsubst:

#!/bin/sh RENVSUBST=${RENVSUBST_PATH:-$(which renvsubst)} if [ -z "$RENVSUBST" ]; then echo "Error: renvsubst command not found. Please install renvsubst from https://github.com/containeroo/renvsubst/releases or set RENVSUBST_PATH." >&2 exit 1
fi $RENVSUBST --prefix="K_" --prefix="CI_" -i -

1. Configure Custom Resource:

Define a custom resource (renvsubst.transformer.yml) next to your kustomization.yml using it:

apiVersion: transformers
kind: renvsubst
metadata: name: renvsubst

1. Integrate with Kustomize:

Update your kustomization.yml to include the custom resource:

resources: - ... transformers: - renvsubst.transformer.yml

By following these steps, you can enhance Kustomize’s capabilities to handle dynamic environments and complex configuration needs effectively.

While Kustomize offers simplicity in managing Kubernetes manifests, its default limitations in variable substitution can be overcome through strategic customization. By leveraging custom transformers and renvsubst, Kubernetes deployments with customize can become more adaptable to diverse and evolving environments, ensuring smoother and more efficient operations.

This article was enhanced with the assistance of an AI language model to ensure clarity and accuracy in the content, as English is not my native language.