AWS Roles. When and how I can use them?


Unless you work via AWS Management Console, in order to access AWS resources you talk to AWS API. All API requests that you make need to be signed by secret access keys (access key ID and secret access key), so that AWS could identify who is making the request and prevent strangers from accessing your resources.

If you ever worked with tools like AWS CLI and Terraform which allow you to manage your AWS infrastructure, then you know that they require AWS credentials in order to work, because they work by making API requests to AWS API. When you work with these tools, you usually put your credentials to a special .aws folder under your user, and they simply take it from there every time you use them.

But what do you do when you need to provide access to AWS resources to some scripts or an application running on your EC2 instance? For example, your application running on an EC2 instance might need to save or fetch some files from an S3 bucket. Do you just put user credentials on that instance in this case? Although this will work, there’s a better way to do this…

For cases like this, when one AWS service needs to access another, AWS offers a special IAM entity that is called a role.

Roles are similar in concept to users. It’s an identity to which you attach policies with permissions to access AWS resources. The difference is that, unlike a user, role is not associated with a single person, instead it is meant to be assumable by any person or any service that needs it.

What roles bring to the table:

  1. Dynamic credentials. Unlike users, roles don’t have any credentials associated with them. Instead, credentials are created dynamically and provided to a user or a service that assumes the role. This saves us some trouble of having to distribute credentials.
  2. Temporary credentials. Each set of credentials that you use requires a rotation. Great thing about roles is that credentials that come with it are rotated regularly for you. If you chose to put user credentials on your machines, the rotation would mean creating a new user account and running configuration management tools to update credentials on all of your servers. This would mean extra work.

To sum it up, EC2 instances that have an IAM role attached automatically have AWS security credentials available which are also regularly rotated for you. Thus, your application or any script that you run on that instance can use those credentials to access AWS resources via API. And you, the person who manages the AWS infrastructure, are saved quite a bit of time and trouble related to security credentials rotation and distribution. Sounds cool? Let’s see how it works.

Example

We will first create an IAM role and attach a policy that gives access to an S3 bucket. Then we launch an EC2 instance with this role attached, ssh into the instance, and do some tests like uploading and downloading files from S3.

When creating a role via Amazon Management Console, it does some steps for you automatically, that’s why to show you how this process really goes I will use AWS CLI.

1. We’ll start by creating a file with our trust policy. This is one of the two types of policies we’ll need to create. A trust policy simply describes who can assume this role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "ec2.amazonaws.com"},
      "Action": "sts:AssumeRole"
    }
  ]
}

In the principle element we specify a user, AWS account, AWS service, or other principal entity that is allowed or denied access to a resource. In this trust policy, we say that we want to allow EC2 service (our instances) to assume the role which in turn means retrieving temporary credentials from AWS STS.

2. After we’ve created a file with our trust policy, we’ll create the role itself.

$ aws iam create-role --role-name logsbucket-role --assume-role-policy-document file://trust-policy.json

I provide the path to a file with the trust policy after the file:// prefix.

3. Now that we’ve created a role and defined who can assume it (the principle), we need to add a set of permissions to our role. We’ll create a permissions policy which defines what actions and resources the principal is allowed to use.

First, I’ll create a file (logs-bucket-permissions.json) with my access policy:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
              "s3:PutObject",
              "s3:GetObject"
          ],
          "Resource": [
              "arn:aws:s3:::artem-server-logs/*"
          ]
      }
  ]
}

Note, that each resource in AWS is uniquely identified by the Amazon Resource Name (ARN), so we need to specify ARNs of those resources we want to make accessible through this role.

In this policy, we allow to get objects from and put object to my test bucket artem-server-logs which I created beforehand.

4. Then we have two options: either to embed this policy in our role making it inline policy (put-role-policy command) or create a standalone (managed) policy and then attach it to the role (attach-role-policy command). The difference between managed and inline policies is explained in full detail here.

In this example, we’ll use inline policy. This way it will be deleted along with the role when I delete the role.

$ aws iam put-role-policy --role-name logsbucket-role --policy-name LogsBucketPermissions  --policy-document file://logs-bucket-permissions.json

5. We’ve created our role and given it permissions, but there is still one thing we need to do before we can use it with EC2 instances. The problem is that we cannot directly attach the role to an EC2 instance, because the application that is running on it is abstracted from AWS by the virtualized operating system (read more on this here). To assign a role to an instance and make it available for the application or script that is running on it, we need to put the role in a container that is called instance profile.

I took a picture from this post where they did a good job explaining how roles work with EC2 instance.

200x200

This picture shows you that instance profile is just a container that you attach to an EC2 instance and that holds your role. An application running on the instance has access to the role through this container and uses the role to retrieve temporary credentials for signing API requests.

When you create a role via Amazon Management Console, the instance profile is created for you automatically. It has the same name as the role. But it doesn’t work this way with AWS CLI, so we need to create a container ourselves:

$ aws iam create-instance-profile --instance-profile-name logsbucket-profile

and then put our role in this container:

$ aws iam add-role-to-instance-profile --instance-profile-name logsbucket-profile --role-name logsbucket-role

You can see what role the instance-profile contains by running:

$ aws iam get-instance-profile --instance-profile-name logsbucket-profile

Things to note: instance profile can contain only one role, but a single role can be included in multiple instance profiles.

6. We’re done with the role now. The only thing left is to lauch an EC2 instance with our role and test it :)

I will launch Ubuntu 16.04 instance via Managements Console since AWS CLI command seems to be too long to paste it here. In the third step of the launch wizard, you can specify which role to attach to the instance.

200x200

As you can see from the picture, instead of roles you actually choose an instance profile that holds your role.

7. After instance has been launched, I ssh into it and check if I can retrieve temporary security credentials that come with the role.

The security credentials are retrieved from the instance metadata. The instance metadata is basically all the data about the instance that is accessible from within the instance itself. So if an application needs to know some information about the instance it’s running on, including temporary security credentials which are available to the instance, it needs to send an HTTP GET requests to the following URL:

$ curl http://169.254.169.254/latest/meta-data/<metadata-category>

Thus, if I want to test whether the security credentials are available to me from within the instance, I can try to get them from the instance metadata:

$ curl http://169.254.169.254/latest/meta-data/iam/security-credentials/logsbucket-role

Most of the times you will use AWS SDK for making AWS API requests. In this case, you don’t have to make requests to the instance metadata to retrieve AWS credentials, AWS SDK does this for you.

AWS CLI is a command line tool built on top of the AWS SDK for Python. It’s used often to manage AWS resources via API. We’ll use it for our tests.

I’ve just tested if I could retrieve security credentials from metadata, now I want to check if I can actually get files from and save files to my test S3 bucket.

I will install AWS CLI on my ubuntu instance:

$ sudo apt-get -y update
$ sudo apt-get -y install awscli

Then I do my testing. I create a file, upload it to my test bucket, delete my local copy and download it again from the bucket.

200x200

It works as expected, but you might notice that I also had to specify the region of my bucket through the environment variable.

Conclusions

In this post we’ve seen how AWS Roles work and what benefits we get from using them.

AWS Roles are used all the time in many different cases.We tried uploading a test text file to an S3 bucket and downloading it from it, but the same way we could archive and upload system and application logs to a remote storage. The application that you run on your EC2 instance might need to access DynamoDB (NoSQL database) which is another AWS service and we see the same problem with credentials management which we now know is easily solved by using roles. Thus, it’s really important to understand how AWS roles work, so that we’re not afraid of using them :)

Here is another interesting link to the video that shows you how AWS roles could be used with SDK-based applications.