Reusable Patterns in CloudFormation
At DEPT, we use a variety of tools to provision infrastructure in the cloud. In this post, we take a look at some of the reusable patterns we've developed using AWS CloudFormation.
At DEPT, we use a variety of tools to provision infrastructure in the cloud. In this post, we take a look at some of the reusable patterns we've developed using AWS CloudFormation.
What is CloudFormation?
If you've ever created infrastructure and/or resources in AWS then there's good chance you've used or at least heard of CloudFormation. For those who are unfamiliar, CloudFormation is an AWS service that allows you to provision and configure almost all AWS resources using yaml (or json) templates.
Here's an example template:
AWSTemplateFormatVersion: '2010-09-09' | |
Description: Deploy a service on AWS Fargate, hosted in a private subnet. | |
Parameters: | |
ENV: | |
Description: Name of the environment | |
Type: AWS::SSM::Parameter::Value<String> | |
Default: /cloudformation/parameters/env | |
VPCStackName: | |
Type: AWS::SSM::Parameter::Value<String> | |
Default: /cloudformation/parameters/vpc/stackname | |
Description: The name of the parent VPC networking stack that you created. Necessary | |
to locate and reference resources created by that stack. | |
ECSStackName: | |
Type: AWS::SSM::Parameter::Value<String> | |
Default: /cloudformation/parameters/ecs/stackname | |
Description: The name of the parent ECS stack that you created. Necessary | |
to locate and reference resources created by that stack. | |
ServiceName: | |
Type: String | |
Default: "CHANGE-ME" | |
Description: A name for the service | |
ContainerPort: | |
Type: Number | |
Default: 8443 | |
Description: What port number the application inside the docker container is binding to | |
ContainerCpu: | |
Type: Number | |
Default: 2048 | |
Description: How much CPU to give the container. 1024 is 1 CPU | |
ContainerMemory: | |
Type: Number | |
Default: 4096 | |
Description: How much memory in megabytes to give the container | |
DesiredCount: | |
Type: Number | |
Default: 1 | |
Description: How many copies of the service task to run | |
Resources: | |
LogGroup: | |
Type: AWS::Logs::LogGroup | |
Properties: | |
LogGroupName: { Ref: ServiceName } | |
RetentionInDays: 365 | |
ECRRepo: | |
Type: AWS::ECR::Repository | |
Properties: | |
RepositoryName: {"Fn::Sub" : "my-test-org/${ServiceName}"} | |
# The task definition. | |
TaskDefinition: | |
Type: AWS::ECS::TaskDefinition | |
Properties: | |
Family: { Ref: ServiceName } | |
Cpu: { Ref: ContainerCpu } | |
Memory: { Ref: ContainerMemory } | |
NetworkMode: awsvpc | |
RequiresCompatibilities: | |
- FARGATE | |
ExecutionRoleArn: {"Fn::ImportValue" : {"Fn::Sub" : "${ECSStackName}-ECSTaskExecutionRole"}} | |
TaskRoleArn: { Ref: TaskRole } | |
ContainerDefinitions: | |
- Name: { Ref: ServiceName } | |
Environment: | |
- Name: "APP_ENV" | |
Value: { Ref: ENV } | |
- Name: "AWS_DEFAULT_REGION" | |
Value: { Ref: "AWS::Region" } | |
- Name: "PORT" | |
Value: { Ref: ContainerPort } | |
Image: { "Fn::Sub": "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ECRRepo}:${ENV}" } | |
PortMappings: | |
- ContainerPort: { Ref: ContainerPort } | |
LogConfiguration: | |
LogDriver: "awslogs" | |
Options: | |
awslogs-group: { Ref: LogGroup } | |
awslogs-region: { Ref: "AWS::Region" } | |
awslogs-stream-prefix: { Ref: ENV } | |
# The ECS service. | |
Service: | |
Type: AWS::ECS::Service | |
DependsOn: LoadBalancerRule | |
Properties: | |
ServiceName: { Ref: ServiceName } | |
Cluster: {"Fn::ImportValue" : {"Fn::Sub" : "${ECSStackName}-ClusterName"}} | |
LaunchType: FARGATE | |
DeploymentConfiguration: | |
MaximumPercent: 200 | |
MinimumHealthyPercent: 100 | |
DesiredCount: { Ref: DesiredCount } | |
NetworkConfiguration: | |
AwsvpcConfiguration: | |
SecurityGroups: | |
- {"Fn::ImportValue" : {"Fn::Sub" : "${ECSStackName}-FargateContainerSecurityGroup"}} | |
Subnets: { "Fn::Split": [ "," , {"Fn::ImportValue" : {"Fn::Sub" : "${VPCStackName}-PrivateSubnetList"}}]} | |
TaskDefinition: { Ref: TaskDefinition } |
view rawfargate-service-template.yml hosted with ❤ by GitHub
The template above creates a simple ECS (EC2 Container Service) resource with one ECS Fargate container task.
The template itself isn't revolutionary but there are few references within the template that make structuring reusable templates easy.
Pseudo Parameters and Intrinsic Functions
Pseudo parameters are basically aliases for common AWS specific configuration data. Utilizing pseudo parameters is critical for composing AWS account and regionally agnostic templates.
Intrinsic functions are helpers that resolve different bits of data within a template. This is useful for a number of things including importing values from other CloudFormation stacks or applying simple string substitutions.
Both pseudo parameters and intrinsic functions can be used together to dynamically resolve more complex configs. Here are some examples.
Render the AWS region using the Ref
intrinsic function and the pseudo parameter AWS::Region
:
- Name: "AWS_DEFAULT_REGION"
Value: { Ref: "AWS::Region" }
Simple string substitution using the Fn::Sub
intrinsic function and the pseudo parameter AWS::AccountId
and AWS::Region
:
Image: { "Fn::Sub": "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ECRRepo}:${ENV}" }
Import a value from the ECS Cluster CloudFormation Stack using the Fn::ImportValue
intrinsic function:
Cluster: {"Fn::ImportValue" : {"Fn::Sub" : "${ECSStackName}-ClusterName"}}
Import and split a list of subnets from the VPC CloudFormation Stack using the Fn::ImportValue
and Fn::Split
intrinsic functions:
Subnets: { "Fn::Split": [ "," , {"Fn::ImportValue" : {"Fn::Sub" : "${VPCStackName}-PrivateSubnetList"}}]}
ResolvedValue Parameters
A more recent feature we've leveraged in CloudFormation templates is ResolvedValue parameters. This special parameter type allows for referencing parameter values in the the AWS Systems Manager (SSM) Parameter Store.
From the example template above:
ENV:
Description: Name of the environment
Type: AWS::SSM::Parameter::Value<String>
Default: /cloudformation/parameters/env
This renders the value of the SSM Parameter located at the param store path /cloudformation/parameters/env
.
By leveraging SSM parameter values in CloudFormation templates, you can easily separate environments across multiple AWS accounts and re-use the same CloudFormation template to provision infrastructure in each account without having to alter or override any template parameters!
Final thoughts
By putting each of these features to use, composing reusable account and regional agnostic CloudFormation templates becomes trivial.
This is extremely useful and efficient if you'd like to create isolated and identical environments (think development, QA, staging, production, etc) in separate AWS accounts.
If building cloud infrastructure and supporting tools interests you, please reach out! We're always looking for talented and impassioned software engineers to work with at Rocket.