Terraform: The Magic Tool for Infrastructure as Code

Terraform is an open-source infrastructure as code software tool that helps organizations manage their infrastructure in a safe and efficient way. It is a tool built by HashiCorp and is used to automate the deployment of infrastructure, such as virtual machines, containers, and network resources, on cloud platforms like AWS, Azure, and Google Cloud Platform.

In this guide, we will go over the basics of Terraform, how it works, how to use it, and some best practices to follow.

Here are the topics that we'll cover in this post:

1. The Basics of Terraform
2. How Terraform Works
3. Getting Started with Terraform
4. Terraform Providers and Modules
5. Advanced Terraform Concepts
6. Best Practices for Using Terraform
7. Potential Issues of Terraform
8. Using Terraform to Provision a Kubernetes Cluster on AWS (A Real Scenario)

Let's dive into each of these topics in more detail.

The Basics of Terraform

Terraform is an open-source infrastructure as code (IAC) tool that helps deploy and manage infrastructure resources across different cloud providers such as AWS, Microsoft Azure, and Google Cloud Platform.

At its core, Terraform simplifies the process of configuring and launching infrastructure resources by using a declarative language to define and automate infrastructure workflows. This means that instead of manually configuring resources through the GUI or CLI interfaces, you can write code that specifies what resources you need, how they should be created, and how they should relate to each other.

Some scenarios where Terraform can be used are:

Creating and managing cloud resources: Terraform can be used to automate the creation and configuration of cloud resources like virtual machines, storage, and network resources. With Terraform, you can easily deploy and update these resources across multiple cloud providers and environments.

Managing infrastructure as code: Terraform can be used to manage infrastructure as code, which means that your infrastructure is version controlled, tested, and maintained just like any other software code. This makes it easier to collaborate on infrastructure projects, track changes, and maintain consistency across environments.

Automating DevOps workflows: Terraform can be used to automate DevOps workflows such as deployments and testing. This makes it easier to manage deployments across different environments like staging and production, and ensures that all deployments are consistent and repeatable.

Provisioning infrastructure for Kubernetes clusters: Terraform can be used to provision the infrastructure resources required for running a Kubernetes cluster, including virtual machines, storage, and networking resources.

How Terraform Works

When you run Terraform, it processes your configuration files to create a plan for the desired infrastructure state. This includes which resources need to be created, updated, or deleted to achieve the desired state. The plan is presented to you so that you can review it before applying any changes to your infrastructure.

Once you approve the plan, Terraform executes it by communicating with the cloud provider's API using the provider plugins. This involves sending requests and receiving responses that allow Terraform to create, modify, and delete resources as necessary.

As Terraform makes changes to your infrastructure, it updates the state file to reflect the current state of your resources. This ensures that Terraform only makes changes that are necessary to achieve the desired state, and can avoid unintended changes that would break your infrastructure.

When you run Terraform again in the future, it reads the state file to determine the current state of your resources, and generates a new plan that takes into account any changes that have been made outside of Terraform. This allows you to manage your infrastructure configuration in a declarative manner, rather than having to manually keep track of the current state of your resources.

Getting Started with Terraform

Before getting started with Terraform, you need to make sure it's installed on your local machine. Here's how you can install Terraform:

1. Download the appropriate package for your operating system from the official Terraform website.
2. Extract the downloaded file using your preferred compression tool. For example, if you're on macOS, open the .zip file in Finder and drag the extracted terraform binary in your /usr/local/bin directory.
3. Verify that Terraform is installed correctly by running the following command in your terminal: $ terraform version This should output the version of Terraform that you just installed. 🤩🤩🤩

Now that Terraform is installed 😍, let's create a directory for our Terraform code and create our first Terraform file. In this example, we'll be using AWS as our cloud provider.

Create a new directory in your preferred location. For example, in your home directory:

$ mkdir my_terraform_project

Change into the newly created directory:

$ cd my_terraform_project

Initialize Terraform by running the following command:

$ terraform init

This will initialize a new Terraform working directory and create a .terraform directory in your project directory.

Now, create a new file called main.tf in your my_terraform_project directory. This file will contain the AWS provider configuration and your infrastructure code.

$ touch main.tf

Open main.tf using your preferred text editor. Add the following code:

provider "aws" {
  access_key = "<your-aws-access-key>"
  secret_key = "<your-aws-secret-key>"
  region = "us-east-1" // Optional: Replace with your desired region
}

This code specifies that we will be using AWS as our provider in the us-east-1 region.

Next, we can start creating our infrastructure by adding resources to our main.tf file. For example, let's create an S3 bucket resource. Add the following code to main.tf:

resource "aws_s3_bucket" "my_bucket" {
  bucket = "my-awesome-bucket"
}

This code creates an S3 bucket with the name my-awesome-bucket.

Save `main.tf` and run the following command in your terminal:

$ terraform plan

This will create an execution plan and show you what Terraform will do when you apply your configuration.

Finally, to create your infrastructure, run the following command:

$ terraform apply

Terraform will prompt you to confirm if you want to create the resources described in your configuration. Type yes and hit enter to confirm.

Congratulations, you have now created your first infrastructure using Terraform! 🎉🥳🎉🥳

Terraform Providers and Modules

Terraform Providers and Modules serve as the backbone of any successful infrastructure deployment. Let's first dive into the importance of Providers.

Terraform Providers are used to interact with various APIs and services, including AWS, Azure, Google Cloud, and others. Providers make it possible for Terraform to create, manage, and delete resources and services in these platforms. They essentially act as a bridge between the Terraform code and the platform or service being interacted with.

Terraform comes with many built-in providers, which are included in the installation package. These providers are then initialized in the Terraform configuration using a provider block. For example, for AWS, the following code would be used to initialize the provider:

provider "aws" {
  region = "us-west-2"
}

This would then grant access to all the resources in the AWS us-west-2 region.

However, Terraform also supports custom providers, which can be written in any language that can communicate with APIs and services. This is useful for interacting with a platform or service that doesn't have a built-in provider in Terraform.

For example, if you wanted to interact with a custom API, you could write a custom provider for that API and configure it in the Terraform configuration. This would allow you to create, manage, and destroy resources in that API using Terraform.

provider "mycustomapi" {
  api_key = "xxxxx"
  base_url = "https://mycustomapi.com/v1/"
}

To create a custom provider, you will need to follow some conventions. These can be found in the official Terraform documentation.

Now, let's talk about Terraform Modules.

Terraform Modules are a way to organize and share code across multiple projects. Modules can be thought of as reusable components that can be used across different Terraform configurations.

Modules contain groups of resources that can be instantiated by calling the module in a Terraform configuration. By abstracting these resources into modules, it's easier to manage and apply changes across multiple projects and environments.

Terraform includes a number of pre-built modules that can be used in Terraform configurations. These modules can be located on the Terraform Registry. The Terraform Registry's community-driven model makes it possible to extend Terraform's functionally by adding pre-built modules that can be shared among teams.

For example, to use the AWS VPC module from the Terraform Registry, you would write the following code in your Terraform configuration:

module "aws_vpc" {
  source = "terraform-aws-modules/vpc/aws"
  version = "2.9.0"
  
  name = "my-vpc"
  cidr = "10.0.0.0/16"
}

This would import a module from the Terraform Registry and use it to create an AWS VPC.

Advanced Terraform Concepts

One of the key features of Terraform is its ability to manage infrastructure through code. This means that you can define the resources you want to create, modify or delete in a declarative configuration file, check that file into version control, and run Terraform commands to apply those changes to your infrastructure.

However, as your infrastructure grows and becomes more complex, it can become difficult to manage everything in one configuration file. This is where remote state comes in. Remote state allows Terraform to store the state (current status) of your infrastructure on a remote backend instead of locally on your machine. This means that multiple team members can work on the same infrastructure at the same time, and that you can maintain a consistent state across different environments.

To enable remote state, you specify a backend in your Terraform configuration file. This backend can be any supported storage provider, including S3, Azure Blob Storage, or even a hosted solution like Terraform Cloud. Once you configure your backend, Terraform will automatically store the state there after each change.

Here's an example of how you might configure Terraform to use an S3 bucket for remote state:

terraform {
  backend "s3" {
    bucket = "my-terraform-state-bucket"
    key    = "terraform.tfstate"
    region = "us-east-1"
  }
}

Next, let's move on to backends. A backend is simply a way to store Terraform state outside of the local file system. Using a backend allows multiple team members to work on the same Terraform project easily, since it enables team members to share the state file.

Backends also allow you to take advantage of features like locking, which prevents conflicting changes from occurring simultaneously. Terraform Cloud is one such backend provider that offers a rich feature set, including version control, notifications, and team management.

Here's an example of how you might configure Terraform to use the Terraform Cloud backend:

terraform {
  backend "remote" {
    organization = "my-terraform-org"
    workspaces {
      name = "my-workspace"
    }
  }
}

Finally, let's talk about workspaces. Workspaces allow you to manage multiple environments within the same Terraform codebase. For example, you might have a production environment and a staging environment, and you need to manage both with the same codebase.

Workspaces allow you to do this by creating separate instances of state for each environment. Each workspace can have its own configuration variables, and you can switch between them easily.

Here's an example of how you might create a new workspace and switch to it in Terraform:

$ terraform workspace new staging
$ terraform workspace select staging

These advanced Terraform concepts allow you to work more effectively and collaboratively with Terraform. By using remote state, you can store critical information safely outside the local file system. Backends enable team members to work on the same project seamlessly, and workspaces make it easy to manage multiple environments with the same codebase.

Best Practices for Using Terraform

Here are some more best practices for using Terraform:

Use variables

Variables help you define your infrastructure resources in a more flexible way. You can use variables to define the values of your resources, such as the size and type of instances, and easily change them as your needs evolve. Here is an example of how to use variables in Terraform:

variable "region" {
  default = "us-east-1"
}

provider "aws" {
  region = var.region
}

resource "aws_instance" "web_server" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
}

In this example, the region variable is defined with a default value of us-east-1. It is then used in the AWS provider configuration to set the region. Finally, the aws_instance resource is defined with a fixed ami value but a variable instance_type.

You can override the default value of the region variable when running the terraform command by using the -var flag, like so:

$ terraform apply -var="region=eu-west-1"

Use sensitive data handling

Terraform allows you to manage sensitive data, such as passwords and access keys, in a more secure way. You can use the sensitive argument to mark certain data as sensitive, which will prevent it from being displayed in the logs or state file. Here is an example:

resource "aws_db_instance" "my_db" {
  engine               = "mysql"
  engine_version       = "5.7"
  instance_class       = "db.t2.micro"
  allocated_storage    = "10"
  name                 = "my_db"
  username             = "admin"
  password             = random_password.password.result
  port                 = 3306
  parameter_group_name = "default.mysql5.7"
  multi_az             = false
  skip_final_snapshot  = true
  tags                 = {
    Environment = "dev"
  }
}

resource "random_password" "password" {
  length           = 20
  special          = true
  override_special = "_%@"
  min_upper        = 2
  min_lower        = 2
  min_numeric      = 2
  min_special      = 2
}

output "db_password" {
  value     = aws_db_instance.my_db.password
  sensitive = true
}

In this example, a random password is generated using the random_password resource. This password is then used in the aws_db_instance resource as the password argument. Finally, the password is exposed as an output, but marked as sensitive. This means that when you run the terraform apply command, the password will not be shown in the console or written to the state file.

Use Terraform state management

As mentioned earlier, Terraform uses a state file to keep track of the current state of your infrastructure. This file contains information such as the resources that have been created and their current status. It is important to manage this file carefully to prevent accidental changes. One way to do this is to use remote state management, which stores the state file in a secure location, like a remote S3 bucket. Here is an example of how to use remote state management:

terraform {
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "my-terraform-state"
    region = "us-east-1"
  }
}

resource "aws_instance" "web_server" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
}

In this example, the S3 backend is configured using the terraform block. This tells Terraform to store the state file in an S3 bucket called my-terraform-state. The aws_instance resource is then defined as usual. When you run the terraform apply command, Terraform will upload the state file to the S3 bucket and keep it up-to-date as changes are made.

Using these best practices will help you write more flexible and secure Terraform code that is easier to maintain and collaborate on.

Potential Issues of Terraform

While Terraform offers numerous benefits to infrastructure management, it also has some potential issues. One major concern is the complexity of the codebase and the learning curve required to master the tool. This can be especially challenging for teams with little experience in infrastructure automation or those without a strong background in programming. Terraform also has some limitations when it comes to certain cloud infrastructure providers. While most of the major cloud providers such as AWS, Azure, and Google Cloud are supported, Terraform may not integrate with some smaller or less popular infrastructure providers.

Another concern is the state management approach used by Terraform. While Terraform stores infrastructure state information in a file, it can be prone to errors if multiple teams are working on the same project at the same time. Additionally, certain changes to infrastructure may require additional manual work which can slow down the automation process.

Using Terraform to Provision a Kubernetes Cluster on AWS (A Real Scenario)

We've already explored the advantages of Terraform. Now, we'll delve into a practical application of Terraform by illustrating how it can be used to set up a Kubernetes cluster on AWS.

Prerequisites:

• An AWS account
• Terraform installed on your local machine
• kubectl installed on your local machine

Steps:

1. Create a directory structure for your project:

$ mkdir aws-k8s-terraform
$ cd aws-k8s-terraform

1. Create a file named main.tf with the following contents:

# initializes the provider for AWS and sets the region
# to us-west-2.
provider "aws" {
  region = "us-west-2"
}

# creates a Virtual Private Cloud (VPC) with the CIDR block of 10.0.0.0/16.
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

# creates a public subnet within the VPC with the CIDR block of
# 10.0.1.0/24 and sets the VPC ID to the ID of the VPC created
# in the previous block. It also gives a name to the subnet
# called "public".
resource "aws_subnet" "public" {
  cidr_block = "10.0.1.0/24"
  vpc_id     = aws_vpc.main.id
  tags = {
    Name = "public"
  }
}

# creates an internet gateway and associates it with the VPC
# created in the first block by setting the VPC ID to the
# ID of the VPC.
resource "aws_internet_gateway" "gw" {
  vpc_id = aws_vpc.main.id
}

# creates a public route table and sets the VPC ID to the ID
# of the VPC created in the first block.
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id
}

# creates a route within the public route table that directs all
# traffic to the internet gateway created in the fourth block.
resource "aws_route" "public_internet_gateway" {
  route_table_id         = aws_route_table.public.id
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = aws_internet_gateway.gw.id
}

# associates the public subnet created in the third block with the
# public route table created in the fifth block.
resource "aws_route_table_association" "public_subnet" {
  subnet_id      = aws_subnet.public.id
  route_table_id = aws_route_table.public.id
}

# creates a security group for the Kubernetes cluster and
# allows all traffic over TCP on any port and all egress traffic. 
resource "aws_security_group" "k8s_cluster" {
  name_prefix = "k8s-cluster"
  ingress {
    from_port   = 0
    to_port     = 65535
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

# creates the Kubernetes cluster using a module called "kubernetes".
# enables cluster autoscaling, sets tags and uses the security group
# created earlier.
module "kubernetes" {
  source = "terraform-aws-modules/kubernetes/aws"
  name_prefix = "k8s"
  subnets     = [
    aws_subnet.public.id
  ]
  vpc_id                    = aws_vpc.main.id
  node_instance_type        = "t2.medium"
  ssh_key_name              = "my-ssh-key"
  cluster_autoscaler_enable = true
  tags = {
    Terraform   = "true"
    Environment = "dev"
  }
  node_security_group_ids = [
    aws_security_group.k8s_cluster.id
  ]
}

1. Initialize Terraform:

$ terraform init

1. Preview the changes that Terraform will make:

$ terraform plan

1. Apply the changes:

$ terraform apply

Once the cluster is created, get the kubeconfig file:

$ aws eks update-kubeconfig --region=us-west-2 --name=k8s-cluster

Verify that you can connect to the Kubernetes cluster:

$ kubectl get nodes

Output:

NAME                                         STATUS   ROLES    AGE     VERSION
ip-10-0-1-100.us-west-2.compute.internal      Ready    <none>   2m28s   v1.18.9-eks-d1db3c

Congratulations! You have successfuly created a Kubernetes cluster on AWS using Terraform. This cluster can now be used to deploy and manage applications in a scalable and efficient manner.

Conclusion

Terraform provides a great solution for managing infrastructure in a seamless way. By following the best practices outlined in this post, you can ensure that your infrastructure is managed in a safe and efficient manner. With Terraform, you can easily create, modify, and delete resources on any cloud platform that is supported.

Terraform integrates seamlessly with many other tools and services, such as version control systems, monitoring systems, and CI/CD pipelines. This makes it an ideal choice for organizations that want to build a fully integrated, end-to-end infrastructure management system.

Terraform is an excellent👌 tool for managing cloud infrastructure. Whether you are just starting out or you have a large, complex infrastructure, Terraform has the features and capabilities to help you manage your infrastructure with ease.