A few tips and lessons for getting started with CloudFormation
It’s important to keep things simple when iterating on any development project. Unfortunately, when a project has multiple contributors, it’s easy for things to get complicated. This blog post will give you some insights on how to set up your team for success and avoid some common pitfalls, all while simplifying your iteration process.
1. Set up a clear directory structure
This is especially important for large and complicated CloudFormation Templates. It’s always a good idea to set standards in the beginning, before things get complicated. While it may seem cumbersome to define a directory structure when first creating a template, it will add clarity for the deployment phase, and for your team. Having a clear directory structure comes in handy for complicated templates with a variety of components—such as AWS Quickstart files, scripts, and multiple sets of parameters.
I suggest a project folder with a clear name so that sharing it using source control and cloning the repository would be straightforward for future developers. Then create subfolders for parameters (I’ll elaborate on this one next), templates, scripts, and other types of files you may need. Here’s an example of a starting point for your directory structure:
This is an example from a small template, but you can imagine the utility of having a space for many users to specify the parameters or tags they need. As your template grows, be sure you are using meaningful names for your various files to improve clarity for your team and your future self.
2. Decouple parameters and tags using separate files
If you’re working alone on a template that you aren’t ever planning to hand off to another party, you might be in the habit of using defaults to populate the parameters in your template.
Using defaults to populate parameters can lead to problems when you have multiple people working on the same template. For example: I’m working with a team to create and iterate on a CloudFormation template which has a variety of parameters and resources, as well as additional elements like PowerShell scripts. I’m testing the stack daily, and so are other members of my team in different environments. Different environments require different parameters and changing parameters in the template each time is not only poor practice, but also not feasible. Enter: decoupling of parameters!
You may already be using JSON to create parameter files using key-value pairs like this, but today I’d like to teach you about my preferred format, the .ini file. It is an extremely clean format which consists only of key=value, each on a new line. I use this for both parameters and tags. Here’s an example:
This format is great because it’s really clear to me what each parameter is and I can change them really easily in one location, without searching through the [often long and cluttered] list of parameters at the top of my template file.
One “gotcha” with .ini files: they don’t play nice with comments or spaces. You may run into issues where you are blocked from passing in parameters which contain spaces. This also means you have to mind your trailing spaces.
As I mentioned above, I also recommend using .ini files for your tags. Tagging your resources is always a good idea, but in the console it can be monotonous to do manually. Using a tags.ini at deployment will tag every taggable resource created by the stack with the same tags. If you need unique tags on a specific resource as well, you can specify those within the template as you do normally, CloudFormation will give that resource both the global tags and its unique tags.
3. Use cloudformation deploy to send it in!
Using the AWS CLI is a simple and powerful way to deploy a CloudFormation stack in AWS. If you need help getting started with the AWS Command-Line Interface, see the documentation for installation and configuration instructions.
You may already be familiar with some of the AWS CLI cloudformation capabilities. Let’s first discuss why I use deploy over create-stack or update-stack.
create-stack is used for—you might have guessed—creating a new CloudFormation stack. update-stack is often employed after a successful initial deployment with create-stack to add changes to the existing stack. Both of these commands share many options, making them fairly straightforward to use in tandem.
deploy, which was introduced later, can either create OR update a stack. deploy requires two arguments, –template-file and –stack-name. If the stack name corresponds to an existing stack, it will update the stack. Otherwise, it will create a new stack*. This makes cloudformation deploy the one command to rule them all (well, not quite, but let me explain)!
One thing to be aware of in a test environment: if the existing stack is in a state of ROLLBACK_COMPLETE, you will receive an error saying the stack cannot be updated. Two easy ways around this are to either delete the failed stack (this usually takes no more than a minute) or to specify a new stack name each time you run deploy.
Unless your use case calls for some of the more obscure arguments available in create-stack or update-stack, deploy will likely meet your needs and simplify things, at least a little. The options that deploy shares with create-stack and update-stack are as follows: –stack-name, –capabilities, –role-arn, and –tags. Two important options that differ in deploy are –template-file and –parameter-overrides.
In the olden days, before you read this blog post, you might have created a stack using something like this:
aws cloudformation create-stack --stack-name myteststack --template-body file://sampletemplate.yaml
–template-file is basically exactly the same as –template-body, except the command already knows it is looking for a file, so you can just enter the path to the file, as follows:
aws cloudformation deploy --template-file /path_to_template/template.yaml
Similarly, –parameter-overrides parallels the function of –parameters, the main difference being that deploy –parameter-overrides will use the existing parameters if none are specified when updating (rather than creating) a stack. One option accepted here that isn’t explicitly stated in the documentation is –region, which comes in handy for many use cases.
Now, here’s where it gets awesome, especially if you’re working with a team. Assuming you’ve convinced your team to follow your directory structure, it is pretty painless for them to alter this command to match their specifications:
aws cloudformation deploy --template-file templates/template.yaml --stack-name myteststack --parameter-overrides $(cat parameters/parameters.ini) --region us-east-1 --tags $(cat tags/tags.ini)
Simply drop this command in the README.md and use it again, and again, and again. This is kind of a hack on the parameters and tags, because I prefer the beauty of the .ini. If you prefer your beauty to be in the command line commands, consider using JSON for your parameters and tags, then deploying like this (without the sub-shell cat):
aws cloudformation deploy --template-file templates/template.yaml --stack-name myteststack --parameter-overrides file://parameters/parameters.ini --tags file://tags/tags.ini
If you have only one or two tags or parameters, it may not be too difficult to enter these inline at deployment, but if you have upwards of 20 parameters which change between iterations like I do, using this methodology will save you time and headaches!
Summary:
The top three top things that can simplify your iteration process are having a clear directory structure, decoupling parameters using .ini files, and deploying to AWS using deploy in the command line interface. If you have questions about CloudFormation, or any other AWS service, let us know! Schedule a consultation with our AWS Experts, or reach out to us at info@1strategy.com. We’d love to chat with you about how 1Strategy can help your business with your journey into the AWS cloud. Happy deploying!
