Automating AWS Organizations with Terraform
In the first part (german) of our article series we explained the basics of terraform. In this part we will implement a real-world scenario: We will configure our AWS account in a way that allows multiple teams to access a wide variety of AWS services, but still isolate their resources from another. Terraform allows us to automate this process, so we will be able to create isolated environments for each team in a matter of minutes.
Scenario
As a company, you might already be using AWS. As the number of employees increases, you might have walked into some issues:
- Developers can see and manage the resources of other teams
- It is not easy to see which projects allocate which resources – and therefore it is difficult to tell how much costs a project causes. Though it is possible to tag resources, it is often not consequently done
- Older projects and their resources are forgotten about but are still paid for
Those issues are solvable with AWS Organizations, which allows you to create AWS sub accounts. People can login to the main account. If they have been assigned the according permissions, they can switch into the sub accounts. The main account will be charged for the costs caused by the sub accounts. The costs can be broken down by sub account.
The configuration of those sub accounts requires some work though. We need to add roles and assign user accounts to those roles to manage permissions.
Goal
We will automate these tasks with Terraform. We will be able to create and configure a complete sub account by adding the following lines to the configuration:
module "my_subaccount" {
source = "./modules/aws-account"
account_name = "my-account-name"
email_address = "my-team@my-company.com"
owner_users = ["some-iam-username"]
dev_users = ["developer-a", "developer-b"]
reader_users = ["junior-developer-x", "manager-y"]
}This will allow us to create AWS account in a matter of minutes.
Concept: How we will structure our AWS accounts
Before starting out with Terraform, we will first draft a concept about what we want to achieve. The following steps describe how a manual configuration via the AWS console can be done, without using Terraform.
Step 1 – Create an AWS sub account
We start by adding a new account to our organization. Navigate to My Organizations and create an account with a globally unique mail address. In the field “IAM role name” you can enter “owner” – the default name is quite long, but you may enter anything you like.
Step 2 – Switching accounts
When you are logged in to the root AWS account, you can click on Switch Roles in the main menu into the new account. You need to enter the Account ID which you can see on the My Organizations page. “Role” ist the name of the role we entered when creating the account (“owner”, if you followed the instructions in step 1).
You may create an Account Alias, so neither you or your colleagues need to remember the ID. Switch to the account and navigate to the IAM Dashboard to do so.
Step 3 – Role concept
The permission management in the sub accounts is done via roles. The role “owner” was automatically added when we created the account. It has full administrative access. New roles can be created at IAM/Roles. When asked about the “Type of trusted entity”, choose “Another AWS account” and enter the ID of the root account. This will allow us to assume the role from the root account. By using policies, we can set the desired permissions for the role.
For example, we create a reader role which gets the ReadOnlyAccess AWS policy assigned, and a dev role, which gets the policies ReadOnlyAccess, AmazonEC2FullAccess and AmazonS3FullAccess.
When everything was set up correctly, you can now use an administator IAM account in the root AWS account to switch to the created roles.
Step 4 – Switching roles
To allow other users to assume roles in the sub accounts, we need to adjust some IAM settings in the root account.
In the **root AWS account*, go to IAM/Policies and create the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::XXXXXXXXXXXX:role/ROLENAME"
}
]
}Replace XXXXXXXXXXXX by the account ID of the sub account, and ROLENAME by the name of the role in the sub account. Create a new Group and assign the policy to the group. Now you will only need to add users to this group which should have the permission to assume that role.
Implementation with Terraform
Terraform Module
Creating and configuring an AWS account requires multiple Terraform resources. Multiple resources can be combined into a single unit using Terraform Modules. Every directory with a terraform configuration is a module. We will create a Terraform module which creates and configures an AWS account. This allows us to create multiple accounts by reusing the module.
Step 0 – Project structure
We will start with a very basic structure which uses a module. The module is located in the directory modules/aws-account.
The directory structure looks like this:
.
├── config.tf
├── main.tf
└── modules
└── aws-account
└── variables.tfYou can look at the content of the files in this GitHub repository. Every commit provides a working example. You can checkout each commit, adjust main.tf by changing the AWS account name and email, run terraform apply, and take a look at the changes in the AWS console.
terraform init initializes the module. It needs to be called if we add a new module block. terraform apply now runs successfully (though we haven’t yet created any resources).
Step 1 – Creating an AWS account
Let’s do something useful and create an AWS Organizations account. We will use the aws_organizations_account resource. The account name and email address are defined by the variables we created in the last step.
If you’ve already used the AWS CLI, Terraform will use the credentials at $HOME/.aws/credentials. If you have trouble with the authentication, head over to the Authentication section in the Terraform Docs
Step 2 – Manage the sub account and create an account alias
Step 1 executed commands in the root AWS account. However, we want to be able to manage both, the root and the sub account. To achieve this, we will add a second Terraform provider inside of our module. By using the assume_role parameter we can tell Terraform to switch to the sub account (described by the role_arn) before executing actions when using this provider. The provider can be used by adding the provider parameter to any AWS Terraform resource.
We will use this provider to create an IAM account alias. Now it is possible to switch to the account by using the account name instead of the numeric AWS ID.
Step 3 – Adding roles and permissions
Using the resources aws_iam_role and aws_iam_role_policy_attachment, we create roles with different permissions. assume_role_policy is required to allow the root AWS account to use those roles. This was previously configured with the AWS UI, now we are using a JSON document, preprocessed by templatefile so we can insert the ID of the AWS root account.
Step 4 – Switching roles
Though you as a admin in the root account can switch to the roles in the sub accounts, other users can’t. We will change that by adding groups and policies to the root AWS account. We assume that the IAM user accounts were already in your AWS account, and we address them by their username. To be able to define which users can access which role, we add new variables to our module. The definition of the required groups, policies, and assignment of users to the groups are put in roleswitch.tf.
This blog article outlined how AWS Organizations helps solve a specific problem and how it can be automated using Terraform. Terraform facilitates the automation of the AWS Cloud. If you have any further questions about the article/ tutorial, please do not hesitate to contact us.
As part of our eoda | analytic infrastructure consulting, we are your contact for state-of-the-art technologies in the field of Data Science. We would also be happy to support your company in working with modern cloud platforms. Contact us.