- read

Terraform Summary

Daniel Alzueta 58

Infrastructure as code software tool that allow us provision infrastructure in different Cloud Service providers.

Consist of two elements

  • HCL — HashiCorp Configuration Language : Using that we describe the infrastructure that we want to provision
  • Terraform client tool CLI : Use to analyze and execute our Terraform configuration.

Resources:

Are defined by the following elements :

  • properties : in the form property name = value
  • blocks: represent a grouping of properties, such as the site_config block inside the azurerm_app_service resource

Variable Types:

The more common variables are : string, bool, number, null, list (or tuple), map and any.

Use of map example :

variable “tags” {
type= map(string)
description =”Tags”
default ={}
}
variable “app_settings” {
type = map(string)
description = “ App settings of the service”
default = {}
}

Next in the terraform.tfvars file

tags = {
ENV = “dev”
CODE_PROJECT = “test”
}
app_settings = {
KEY = “value of the key”
}

To use this variables in our main.tf file

resource “azurerm_resource_group” “rg” {
name …
tags = var.tags
}
resource “azurerm_app_service” “app”{
name = “${var.app_name}”

app_settings = var.app_settings
}

Use of any type, example (here we will see the use of for instruction):

First, we define the variables in variables.tf file

variable "webapps" {
type= any
description ="List of app services to privision"
}

terraform.tfvars file

webapps = {
webapp1 = {
"name" = "webappexample1"
"location" = "West Europe"
"dotnet_framework_version" = "v4.0"
"serverdatabase_name" = "server1"
},
webapp2 = {
"name" = "webappexample2"
"dotnet_framework_version" = "v4.0"
"serverdatabase_name" = "server2"
}
}

In the main.tf file

resource "azurerm_app_service" "app" {
for_each = var.webapps
name = each.value["name"]
location = lookup(each.value, "location", "West Europe")
...
}
output "app_service_names" {
value [for app in azurerm_app_service.app : app.name]
}

Interesting use case of the type any

Using any we can create blocks to define some resource’s property

example , if we define a variable called rgs_rules as type any and next in terraform.rfvars

rgs_rules = [
{
name = "rule1"
priority = 100
direction = "inbound"
access ="Allow"
....
},
{
name = "rule2"
priority = 120
direction = "inbound"
access ="Denied"
....
}
]

In main.tf

resource "azurerm_network_security_group" "security_group" {
...
dynamic "security_rule" {
for_each = var.rgs_rules
content{
name = security_rule.value["name"]
priority = security_rule["priority"]
....
}
}
}

Basic Structure file:

# Configure the Azure provider
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 2.65"
}
}
required_version = ">= 0.14.9"
}
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "rg" {
name = "myTFResourceGroup"
location = "westus2"
}

Variables : (we can define the variables in a .tfvars file)

variable "name_of_variable" {
description = "description of the variable"
default = "value [it is optional]"
}

Next, we can use this variable in this way :

vars.name_of_variable

How we can pass the variables :

1- using the -var flag when we execute terraform plan command.

terraform plan -var ”var_name=$value”

2- Declaring environment variables

$env:TF_VAR_<var_name>

3- Using an file with .tfvars extension

Local variables :

A local value assigns a name to an expression, so we can use it multiple times within a module without repeating it.

Local values can be helpful to avoid repeating the same values or expressions multiple times in a configuration, but if overused they can also make a configuration hard to read by future maintainers by hiding the actual values used.

Use local values only in moderation, in situations where a single value or result is used in many places and that value is likely to be changed in future. The ability to easily change the value in a central place is the key advantage of local values.

Once a local value is declared, we can use it in expressions as local.<NAME>

locals {
variable_name_local = "${var.var_name}-${var.other_var}-test"
}

How we can use them :

e.g.

resource “azurerm_resource_group” “rg” {
name = “RG-${local.variable_name_local}”
….
}

Output:

when we use an IaC Tools such as Terraform, it is necessary some times retrieve output values after code execution.

In this way, this values can be used next by others programs. Like as we build a CI/CD integration.

for example:

extract of main.tf file


resource "azurerm_app_service" "app" {
name = "${var.app_name}"
location = azurerm_resource-group.rg.location
….
}
output "webapp_name" {
description =" output of the webapp name"
value = azurerm_app_service.app.name
}

// Here we defined a output block specified a name and value, the value is the name of the app service.

Provisioning infrastructure in multiple environments

We could make the following structure of folders for a single environment project. Then, we will execute the classical terraform execution workflow by running :

  • terraform init
  • terraform plan
  • terraform apply
structure of a single terraform project

For a multiple environment project we would create the folders structure that is above.

Then, to provisioning each of the environments, we will execute the following commands

  • terraform init
  • terraform plan -var-file=”<environment-folder>/terraform.tfvars”
  • terraform apply -var-file=”<environment-folder>/terraform.tfvars”
structure for a multiple environment project

Obtaining external data using data block or terraform_remote_state block

When we provisioning infrastructure with terraform, it is sometime necessary retrieve information about the others resources that already exists.

1- to retrieve the information of a service plan that already exists we create the next data block :

data "azurerm_app_service_plan" "myplan" {
name = "app-service-plan-name"
resource_group_name = "rg-name"
}

In this way, now we can use this information to create others resources, e.g.

resource “azurerm_app_service” “app” {

app_service_plan_id = data.azurerm_app_service_plan.myplan.id
...
}

note: when we call the terraform destroy command, this not destroy the resources called by the data block.

2- retrieve information of an external Terraform State file. In this case, we will read outputs of an existing Terraform State file that was used to provisioning resources. It Terraform State must was already deployed.

for example, in the main.tf file of the Terraform project of which we going to read

….
resource "azurerm_app_service_plan" "myplan"{
….. // intructions to create the app service plan
}
// an output defined
output "serviceplan_id" {
description ="Id of the service plan"
value = azurerm_app_service_plan.myplan.id
}

Next, to allows us to retrieve outputs present in another Terraform State File we create the terraform_remote_state block. Here we specified the remote backend information. (That indicate where is located the tfstate).

data “terraform_remote_state” “service_plan_tfstate” {
backend ="azurerm"
config = {
resource_group_name = "rg_tfstate"
storage_account_name= "storstate"
container_name = "tfbackends"
key = "serviceplan.tfstate"
}

Then we can use the output generated

resource “azurerm_app_service” “app”{
name = "${var.app_name}-${var.environment}"
location = azurerm_group_name.rg.name
app_service_plan_id = data.terraform_remote_state.service_plan_tfstate.service_plan_id
}

Note: This technique is very useful to have separate the Terraform configuration that deploys a complex infrastructure.

3- Read from external sources

We can read information to external sources, executing for example a PowerShell script (also, we could use other script languages) that will return the resource group name depending the environment passed to the script.

// then , if we have a script that receive for parameter an environment, and next write-out

… Powershell script
Write-output "{ ""location"" : ""$location"" }" // next of run a process, return the location

Now, we want retrieve this location in our Terraform file. For that we use the external block.

//in the main.tf file

data “external” “getLocation” {
program = ["Powershell.exe", "./GetLocation.ps1"]
query = {
environment = "${var.environment_name}" // using this we pass the parameter to the script
}
}

// next, to use this information received, we can do ..

resource “azurerm_resource_group” “rg” {
name = "RG-${local.resource_name}
location = data.external.geoLocation.result.location
}

// also, we could create a output block to expouse this value. In the other hand, we could use the null_resource block to execute commands also.

Terraform Build-in Functions:

The language supplied with Terraform includes functions that can be used in any Terraform configuration.

Upper : function capitalize everything inside.

format : allow us to format text, we use %s to indicate that is a character string.

merge: take maps or objects and returns a single map or object that contains a merged set of elements from all arguments.

Example:

Upper(format(“RG-%s-%s”, var.app_name, var.environment))

Conditional Expressions:

In Terraform we can use conditional expressions like :

condition ? true assert : false assert

resource “azurerm_resource_group” “rg” {
name = var.environment == "Production" ? upper(format("RG-%s", var.app-name)) : upper(format("RG-%s-%s", var.app-name, var.environment))

}

local_file block

// in the main.tf file

// this create when the plan is executed the myfile.txt file and write the content specified.

resource “local_file” “myfile” {
content= "This is the content text"
filename = "../files/myfile.txt"
}

// next execute

terraform init
terraform plan -out=”app.tfplan”
terraform apply “app.tfplan”

// in other folder create other main.tf file and …

data “archive_file” “backup” {
type=”zip”
source_file = “../files/myfile.txt”
output_path= “${path.module}/archives/backup.zip”
} // with this we obtainthe file created in the before step and create a .zip file

// next navigate to the backup.zip file location and execute

terraform init
terraform plan

Generating passwords:

In some cases, when we create an resource that needs a password, like VMs.

To that we can use the resource “random_password” .

Easy way to provisioning several instances of the same resource:

1- Using the count attribute

resource “azurerm_app_service” “app” {
count = var.n_apps // here we indicate the number that we want
name= “${var.app-name}-${var.environment}-${count.index + 1}”

}

// if we want, we could add a output

output “app_service_names” {
value = azurerm_app_service_app[*].name
}

// in this way, for example if we set n_apps as 3, three instances will be built with its names ends as 1, 2, 3

Interesting use case:

We could do some this …

resource “azurerm_application_insight” “appinsight”{
count = var.use_insight == true ? 1 : 0

}

// if the value is 0, then not will be built the app insight.

Terraform CLI

commands:

  • terraform init : initialize the Terraform context
  • terraform fmt : arrange the code with the correct indentation. (an optional flag is -recursive, that indent the code in subfolders of the current folder.
  • terraform validate : check the code validity.
  • terraform plan : create the plan of the resources to create/update/destroy, also validate. (terraform plan -out=”app.tfplan”)
  • terraform apply: execute the plan created. (terraform apply “app.tfplan”)
  • terraform destroy : destroy all resources tracked in the terraform state file. (if we need destroy a single resource and not the others, we can specify that using target option, terraform destroy -target azurerm_application_insight.miapp_insight)
  • terraform import
  • terraform output
  • terraform graph | dot -Tsvg > graphfile.svg

use case of validate command :

this command is useful in a local environment where we test our terraform configuration, but, also in code integration in a continuous integration pipeline, so as to not execute the terraform plan command if the terraform validate command return syntax error.

for example next of execute the command it is showed :

> terraform validate
> $LASTEXITCODE

Where $LASTEXITCODE return 0 if there is no error, otherwise 1 if there is an error.

Note: It is also possible to get the output of this command in JSON format adding -json option in the validate command. ( terraform validate -json)

Terraform Workspaces

Allow us using the same Terraform configuration build multiple environments.

Each of these configurations will be written to a different Terraform state file and isolated of other configurations.

Workspaces can be used to create several environments of our infrastructure.

Workspace commands:

  • terraform workspace new <name of the environment>
  • terraform workspace select <workspace name>
  • terraform workspace list
  • terraform workspace delete <workspace name>

how works its ?

In the main.tf file , we can have the next code

resource "azurerm_resource_group" "rg" {
name "RG-SERVICE-${terraform.workspace}"
location = "West Europe"
}

Next, to create a new workspace we run the command

terraform workspace new test

Then a new workspace is created with the name test and terraform is automatically placed in this.

To provision the test environment, we run the basic commands

terraform init 
terraform plan -out="outtest.tfplan"
terraform apply "outtest.tfplan"

Now, we could create other workspace to other environment

terraform workspace new prod

to provision this new environment we execute newly the basic terraform commands.

terraform init 
terraform plan -out="outprod.tfplan"
terraform apply "outprod.tfplan"

Graph of dependencies:

for generate this graphic we can use a third-party drawing generation tool Graphviz (you can download here http://graphviz.gitlab.io/download/)

So, to generate the graph we execute the next command

terraform graph | dot -Tsvg > graphfile.svg

We execute the terraform graph command , then , send the result to this graph command to the dot utility (This utility is of Graphviz, that was previously installed). Finally, dot utility will generate the graph file.

Then if we going to the folder of the project, we will see the file with the dependency graph.

Debugging Terraform Configuration

In order to enable the debug mode of terraform we have to set TF_LOG environment variable as TRACE.

In windows Operative System :

$env:TF_LOG=”TRACE”

In this way, when we execute the terraform workflow (init, plan, apply) will show the log to each step of the process.

Also, if we want to store the logs in a file, we can set the environment variable TF_LOG_PATH specifying the path to the log file.

To disable the debug mode, we must set TF_LOG as empty string.

Terraform Modules:

A Terraform module is a Terraform configuration that contains one or more Terraform resource. Once this module be created, it can be used in several Terraform configuration files.

Create a Terraform module locally:

We going to create the following folder structure.

  • MyProject/Modules/WebApp the Terraform configuration to provisioning a resource group, app service, and application insight.
  • MyProject/MyApp : the Terraform configuration that will use the module created in the before step.

MyProject/Modules/webapp/variable.tf

variable "resource_group_name" {
description = "Resource Group Name"
}
variable "location" {
description = "Resource location"
default "WestUs"
}
variable "service_plan_name" {
description = "..."
}
variable "app_name" {
description =" ... "
}

MyProject/Modules/webapp/main.tf

resource "azurerm_app_service_plan" "plan" {
name = var.service_plan_name
location = var.location
resource_group_name = var.resource_group_name
sku {
tier = "standard"
size = "S1"
}
}
resource "azurerm_app_service" "app" {
.....
}
resource "azurerm_application_insight" "insight" {
.....
}

MyProject/Modules/webapp/output.tf

output "webapp_id" {
value = azurerm_app_service.app.id
}
output "webapp_url" {
value = azurerm_app_service.app.default_site_hostname
}

MyProject/MyApp/main.tf , Here we will use the module

resource “azurerm_resource_group” “rg” {
name = "RG_Example_01"
location = "WestUs"
}
module "webapp" {
source = "../Modules/WebApp"
service_plan_name = "miserviceplan"
app_name = "myapptest"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
}
output "webapp_url" {
value = module.webapp.webapp_url
}

Next we execute the Terraform workflow

terraform init
terraform plan -out=app.tfplan
terraform apply app.tfplan

When executing the terraform init command, will get the module’s code and integrate its configuration with that of application.

Remotes Modules:

Terraform has set up a public registry, in this registry there are modules developed by cloud providers, communities, and developers.

We can found the Terraform registry here Browse Modules | Terraform Registry.

In the Terraform registry we can filter the registry needed. Then, in the detail section of the module we can found how use it.

We can use this modules to provision resources easily.

For example we could create a terraform configuration file main.tf

resource "azurerm_resource_group" "example" {
name = "my-resources"
location = "West Europe"
}
module "network" {
source = "Azure/network/azurerm"
resource_group_name = azurerm_resource_group.example.name
address_space = "10.0.0.0/16"
subnet_prefixes = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
subnet_names = ["subnet1", "subnet2", "subnet3"]
subnet_enforce_private_link_endpoint_network_policies = {
"subnet1" : true
}
subnet_service_endpoints = {
"subnet1" : ["Microsoft.Sql"],
"subnet2" : ["Microsoft.Sql"],
"subnet3" : ["Microsoft.Sql"]
}
tags = {
environment = "dev"
costcenter = "it"
}
}

Also, we could publish our modules in the Terraform repository using Github.

Share a module using a PRIVATE Git repository

Often, in enterprises there is a need to create modules without exposing the code publicly by archiving them in GitHub repository. To solve it, we can use a Git repo in Azure Repos, which require authentication to access it.

For that, we need a Azure repos project.

For this example we could create one called test-module-sharing. Then upload the module code to this repo, update the readme document with the module description and commit and push all. Next Add and push a Git tag (git tag v1.0.0).

Finally, in the main.tf file where we want use the remote module we write the next code

resource "azurerm_resource_group" "rg" {
name = "RG_MYAPP_EXAMPLE"
location = "WestUs"
}
module "webapp" {
source "git::https://dev.azure.com/...../.../terraform-module?ref=v1.0.0"
.....
}

Internally, when Terraform execute the init command, Terraform will clone the repo locally.

Note: Using the ref parameter that we put in the module call URL, we can specific the version that we want use.

Executing a script file inside of module:

In some moment we could need execute a script of part of the module. For example, we consider have a module called exescript (then we create a subfolder in the module folder called exescript). In this folder add the file script.sh with the code that we want execute.

Then, add the main.tf file of the module with the next code :

resource "null_resource" "exefile" {
provisioner "local-exec" {
comand = "${path.module}/script.sh"
interpreter = ["/bin/bash"]
}
}

Finally, in the terraform configuration we call this module :

module "exefile" {
source = ".../Modules/exescript"
}

The Terraffile pattern

A Terrafile is a YAML config file that lists all your external module dependencies. In this way, is easier manage the version and location of the modules. You can found more information here.

Terraform Module Generator:

Microsoft built a tool to create easily modules using Yeoman. This tool makes the basic structure of a module with the needed files and other files like a Docker File to tests. More information here.

First we need have installed “yo”

npm install -g yo

install the module generator

npm install -g generator-az-terra-module

To create the module, we goint to the folder that we want and :

yo az-terra-module

Next, after answer some questions the files structure is created to the module.

Testing Terraform configuration files

Terratest is a Go library used to write automated tests for IaC.

You can found examples and documentation here.

Other Terraform Tools:

terraform-docs : Help us to create the module’s documentation.

terratest : Useful to test Terraform configuration files.

References:

Standard Module Structure — Terraform by HashiCorp

Build Infrastructure | Terraform — HashiCorp Learn

Refactor Monolithic Terraform Configuration | Terraform — HashiCorp Learn

How To Structure a Terraform Project | DigitalOcean