Lately, I’ve been working on a project that involves a lot of CloudFormation. This infrastructure-as-code service by AWS allows you to provision resources such as servers, storage, and networking by declaring what is needed in code. You create what are called templates which contain the code and then upload them in order to create a stack based on what is defined in the template. A stack is simply a template that has been deployed.
As you start working with CloudFormation templates, you quickly find that they grow very long and start to become unmanageable. In order to simplify things, it’s a good idea to start breaking things into separate templates just as a software developer breaks up code into classes, functions, etc. Splitting things up also allows division of work while different people and teams work on their respective portion of a project. For example, a networking team may create a template that includes VPCs, subnets, route tables, and VPNs. The systems team, on the other hand, might create a template that provisions servers and storage. With separate templates, however, you frequently find that you need a way for them to refer to each other. Those servers, for example, would have to be placed in the subnets created by that networking template.
About six months ago, AWS announced a new feature called, “Cross-Stack References,” which allows you to connect different CloudFormation stacks together by importing and exporting values as needed. Prior to this feature, you had to manually enter subnet IDs, for example, into your template or provide them as parameters. Nested Stacks, another CloudFormation feature, allowed you to deal with some of these challenges, but they take a different approach and have other requirements that can be inconvenient in some cases.
As mentioned, cross-stack references are simply imports and exports. The following is an example of a subnet ID being exported from a networking template. Notice that you take an existing output and add one additional line that exports it:
In this case, a subnet was created in the Resources section of the template and we are now outputting the subnet ID as SubnetID and then further exporting that output as Networking-SubnetID. This name has to be unique within the region of the AWS account where it exists, and it can only be imported into templates in that same region. In this case, we are using the convention of prefixing our export with the name of the template we’re exporting from. This is simply a pattern we tend to see, but it could really be named whatever you like.
Now in order to import the subnet ID into the systems template to use it when creating an EC2 instance, you use Fn::ImportValue as shown below:
And just like that we have the two templates linked together. Now we can launch the networking template and then the systems template (in that order) without ever updating files to include the generated subnet ID or having to enter it as a parameter.
Below are some observations I’ve made while working with cross-stack references and some various tips/tricks I’ve discovered that I thought I would share:
- You can’t delete stacks that export values if those values are being imported by another stack.
- You can’t change the value that is being exported until it is no longer being imported somewhere.
- You can’t import a value into the parameters section. This was disappointing. In cases where you have a parameter being used multiple times throughout a template, it would be helpful to simply import the value from another template, set it as the default value for the parameter, and then leave all the references to the parameter in place. Unfortunately, those references must all be replaced with the import.
- As you shuffle things around and make improvements to your templates, you might need to change where values are being exported from. For example, you might decide that instance profiles should be provisioned in and exported from a security-specific template. As you update the stack containing the EC2 instance that uses that instance profile, CloudFormation will warn you that the instance requires replacement (the server will be terminated). However, I’ve found that if the resulting value will still be the same, nothing will actually happen. Confirm that the value being imported has not changed and you’ll be able to go ahead with the update without a replacement. The same behavior can be observed when switching from a parameter to an import.
- As you create multiple templates, it can be helpful to create a diagram to keep track of the template interdependencies. You can also see a list of exports by going to CloudFormation -> Exports when on the main CloudFormation page. I’d love to see someone put together some kind of visualization tool at some point.
- Not everything is supported in CloudFormation. An instance profile for an EC2 instance, for example, can be changed in the console but not yet in CloudFormation.
- And lastly, depending on how immutable your infrastructure is, you may find that you have to maintain three separate sources of truth: the actual AWS state (which instance profile is actually being used after making a manual change), CloudFormation’s understanding of things (the last template you uploaded before you changed which instance profile was used), and your source code (which shows the server using the correct instance profile). While it would be ideal to automate everything and never make changes outside of CloudFormation, some systems require a degree of manual intervention. Your source code may represent what the architecture really looks like and what would be provisioned if it had to be deployed in a second environment, but it might not necessarily be submitted to CloudFormation due to the interruptions it would cause, and it doesn’t always include those manual changes that were made via the console or CLI to achieve things not possible through CloudFormation.
These have been a few of the lessons I’ve learned as we’ve worked with cross-stack references. This new feature isn’t perfect, but it’s very powerful and we plan on using it a great deal as we approach new projects.
