Your first project on AWS — ECS
Chapter 2 — create task definition, ECS, EC2, IAM, and security group.
The series
- Chapter 1 — The Basics
- Chapter 2 — Elastic Container Service
- Chapter 3 — Application Load Balancer
- Chapter 4 — S3 and CloudFront
In the previous chapter we looked at the basics of AWS, now let’s dive a little deeper and learn about ECS (elastic container service) but before that, here is what our final architecture would look like.
I am assuming you have some knowledge on what is Docker and how applications are deployed using Docker. You can read anout it here.
We will create a simple SPA in reactjs and host it in the S3 bucket (an object store offering from AWS), this is set up behind cloud front service whose main job is to cache and deliver your content faster but that’s not why we are using it here, we are using it for its ability to set up origins and behaviors, origin simply means the place where content is stored, we have 2 such places, one is our S3 bucket which contains the front end(static content) and second is the load balancer which serves dynamic content from the backend (nodejs docker containers).
Internet gateway helps our nodejs service or any other component inside our VPC to connect to the internet. All AWS services are required to be accessed over the internet.
The elastic container registry or ECR is used to store the docker image of our nodejs service which will be downloaded and hosted on a virtual machine by the ECS. Parameter store allows us to store master configurations for our service like the web.config in.net ( cache duration, URLs, etc.).
Elastic container service
The load balancer service is serving us traffic from the ECS or the elastic container service. In simple words, ECS is a service that will be hosting our nodejs docker containers on a set of virtual machines (also called a cluster). The machines on the cluster will be set up by us but ECS takes care of how to place the docker containers on the cluster based on their CPU and memory requirements or specific machine requirements.
Since the aim of the article is to quickly create your first project on AWS, we are covering only one of the ways ecs can be set up.
As you can see above, ECS takes constraints of running our docker container as input, also called the task definition, and deploys the required number of copies on the cluster. We have also specified that our task must run on a specific ec2 type i.e. t3.micro, so ECS will try to find that system on the cluster.
Elastic container registry (ECR)
Elastic container registry is a docker container registry on AWS which makes it easy for us to store and deploy docker container images. We will use this to store the image of our nodejs application and deploy this on EC2 instances aka ECS container instances using ECS.
How do we add a docker image to ECR? Through GitHub actions. I will share a GitHub repo link with you and you can fork this repo, then we will create an AWS IAM (identity and access management), user, to push our nodejs Docker image to ECR. This will be covered in the next article.
AWS free tier
AWS has a free-tier offering for new sign-ups to AWS services, under this some of the services are provided for free for 1 year with monthly limits, for example, EC2 t2.micro instance is available under free-tier for 750 instance hours per month for 1 year. Anything above 750 instance hours will be charged to your saved credit card.
This project will be completed within the free tier limits and all components are created in US-WEST-2 region.
Let’s create ECR in AWS
Login to your AWS account, search for ECR, and select elastic container registry. Under private repository, select create repository and provide a name for your repo. ECR private repo has a limit of 500 Mb/month under the free tier, you will be charged for anything above that.
Click Save. Our repo to hold docker images is ready. The most important thing here is the URI, we will need this during ECS creation.
Now, let’s first create the EC2 and load balancer since this is required for creating the service.
Prerequisite to create the EC2, create an IAM role
Before we create an EC2 machine, we need to create a role with specific policies which will help our EC2 to be compatible with ECS. This role will be then attached to EC2. A role is nothing but specific permissions given to our virtual machine.
Search and select IAM service, on the left select roles, then click create role, select use case as EC2, click next.
search for AmazonEC2ContainerRegistryFullAccess and select the same. Do the same for the following roles, AmazonECS_FullAccess, AmazonSSM_FullAccess, AmazonEC2ContainerServiceforEC2Role. Click next, provide a name for this role, review the selected roles, and create.
We have selected SSM policy to access the parameter store, this is like a configuration file for your application, we can store things like cache duration, etc. Container registry permission allows EC2 to download the latest image from ECR, the remaining are for ECS.
Finally, let’s create EC2
Search and select EC2, on the dashboard, select instances, and click launch instances, the first step here is to select the AMI or the amazon machine image.
A simple definition of AMI would be a software package that contains required files, os and any other data for creating a virtual machine or an ec2 instance.
Search for ECS and select the ECS optimized AMI, next page shows the costs for AMI, ignore this as software costs are 0 and click continue.
On the next page, select the instance type as t2 micro(1GB memory and 1 CPU) which is free tier eligible (750 instance hours per month), anything above that will be charged at 0.012$/hr in the US-WEST-2 region.
Next, set no of instances to 1, for network select your default VPC and a public subnet, here I have selected us-west-2a. Enable auto-assign public IP.
Set IAM role to the newly created role, next in advanced details, copy the following to the user data field. The ECS agent running on this machine will check the ecs.config file for any configuration details, we need to inform the agent how to register this EC2 by providing the cluster name.
#!/bin/bash
echo “ECS_CLUSTER=mfp-dev-ecs-cluster” >> /etc/ecs/ecs.config
Click next, leave the storage settings as is and click next to set the name tag for our EC2.
Click next to create a security group, as discussed in the previous article, the security group acts as a virtual firewall around this EC2 defining inbound and outbound rules for connections to and from this EC2.
For now, we will add 1 inbound rule allowing only SSH connections to this EC2 and later add another rule once we have our load balancer up and running. Outbound rule by default allows all traffic.
Next, click review and launch, then click launch on the next page. It will ask you to select a key pair that will help you log in to your EC2, if you do not have one already you can create a new one and download the same although we will not be using it. Click launch instances, the instance should be ready in a few minutes. You can check the status from the EC2 dashboard.
Create ECS cluster
Search for ECS and select elastic container service, click create cluster, select networking only and provide a cluster name like this
Click create and your cluster is ready, just does not have any instances yet. We will register a new instance in a while.
Create task definition
As we discussed, task definition is a template used by ECS to deploy our Docker containers, you can add more than 1 docker container in a single task definition and the service will deploy all of them.
Again search ECS and select elastic container service, on the left select task definition and create a new task definition, select launch type as EC2 since we will be running our containers on EC2 machines, then set the task definition name as shown below.
Leave the rest as is, then click add container, set container name and path to the image URI from ECR. We also set a soft limit of 500 Mb which is the amount of memory reserved for this container, the container can request up to all the memory in the EC2 instance or up to a hard limit. We will not specify any hard limit here.
Next, set the container and the host port, the container port is the port on which the container is listening, and the host port is the port on which your application is receiving traffic for example 80 for HTTP. We map port 80 of the host to 8080 on the container, 8080 is the port on which we start our service inside the container (server. listen(port, host) inside server.ts file)
Next, set the health check command, this is just a curl command that the container will run to check the health of the service, /api is an HTTP get endpoint inside the service that just returns 200 which means if the service is running we get a 200 ok from /api.
An Interval of 5 seconds would trigger the health check every 5 seconds, a timeout of 5 seconds for the health check to succeed before calling it a failure, and 5 retries after a failed health check before calling the container unhealthy.
Next, set the below environment variables for the container, environment variables help you set configurations based on environments like dev, prod, etc. This is used for simple settings, for example, which port to start listening on, the rest of the settings can be stored in the AWS parameter store. We will explore that in the final article.
Leave the rest of the fields as-is and click add, then click create to create the task definition. This creates version 1 of our task definition, each time we modify this, it will create new versions.
Create ECS service
Select the cluster you just created, then select service, click create
Fill in the below fields
Launch type is EC2 since we will create an ec2 machine and manually add it to our cluster.
Fargate and external are beyond the scope of this article.
Next, select the task definition that we created and its revision or the version i.e. 1. Select the cluster name and provide a name for this service, the service type can be selected as a daemon. In daemon mode, only 1 task is placed on 1 instance whereas in the replica mode tasks are randomly placed on available ec2 instances and each instance may contain more than 1 task. The no of tasks to run defaults to automatic since we have selected the daemon mode which means that a task will be started once at least one ec2 instance is registered in the cluster. Now, click next, continue clicking next until you see the review page, then click create service.
Now, if we go back and check the service tab in our cluster, it will show an active service with desired and running task count as 0 which is expected since we do not have any machine to run these tasks.
Now, let's go back to the ECS cluster and ecs instances tab
Congratulations, we have just registered an ec2 instance on the ECS cluster, next, check the services tab, it shows desired task 1, running is still 0. Desired is 1 since we selected the daemon mode and now we have 1 ec2 which means 1 task can be run on this machine. Running task is 0 as there is no image in our ECR repository and the service will fail to create a new task.
Click on the service name and then select events, you will see a lot of tasks started, this just means that the service tries to start a task, it fails and the service tries to start it again.
So, let's just stop the service from trying to create new tasks until our ECR image is ready. To do this, go to the ec2 dashboard, select the ec2 we just created and click stop instance.
As there is no ec2, the service will not try to create more tasks. We will start this again once we are ready.
In the next article, we will push the image to ECR, create a load balancer, start ECS service and try calling an API.
If you liked my work, buy me a coffee.