Dev Environments: An Essential Tool for Software Quality
Why Infrastructure as Code isn't just for production deployments.
There are many steps on the road to DevOps maturity. Recently, I've been covering some of the most basic concepts, such as stacks, stages, and Infrastructure as Code. Today. I'll stick to these foundational steps and talk about on-demand dev stacks. I'll focus on why dev stacks are perhaps the most important first step teams can take on their DevOps journey.
Your Application, On Demand
First, let's recap some concepts from my last article. One of the great benefits of moving to a cloud platform like AWS is Infrastructure as Code. With Infrastructure as Code, you can spin up the architecture your application needs - network topology, Web servers, databases, file storage, load balancers, etc. - by programming it.
Before Infrastructure as Code, standing up a new version of an app usually meant manually configuring and tending to every component of the system. It was tedious and error-prone. Defining your architecture in a programming language like Python or in a declarative language like AWS CloudFormation means you can deploy and re-deploy your application over and over, consistently and without fear.
Stacks, Stages, and Environments
Before I dive in, let's get clear on our terminology.
Using Infrastructure as Code, you can deploy a stack - your application plus all its supporting infrastructure - quickly and easily. Once you can deploy a stack, you can deploy multiple stacks - stages - for various purposes - e.g., a single stack for production, plus other stacks for development and testing.
People in software development talk a lot about environments - e.g., production environments vs. dev environments. In our view, "environment" encompasses a specific runtime for your application that may or may not be hosted in the cloud. For many development teams, dev environments reside on a developer's desktop or laptop.
Production Stacks and Dev Stacks
Many teams are drawn to Infrastructure as Code to streamline their production deployments. And indeed, repeatable production employments can greatly enhance application quality. Standing up new production stacks opens the door to numerous advanced deployment strategies such as canary testing and blue/green deployments.
But Infrastructure as Code can improve quality before your team even pushes to production. You can use the same code you use to stand up a production stacks to stand up development stacks as well!
Why a Dev Stack?
Personal dev environments are becoming increasingly standardized with tools such as Gitpod and Codespaces. As your team moves more toward standing up stacks, the difference between personal dev environments and dev stacks starts to fade.
Dev stacks allow development teams to test their changes end to end before they're ever pushed to production. Using Infrastructure as Code, teams are assured that what they're testing is (apart from a few small config changes) identical to what will run in production.
Having a central dev stack for your team is great. However, giving developers their own fully deployed stacks makes it even easier to test changes before they ever hit main.
With individual dev stacks, your developers can deploy individual changes faster. This leads to greater flexibility and reliability over grouping many changes together into a single deployment. In addition, when building on cloud services, testing against a live service is better than attempting to replicate that service on a laptop.
The "Official" Dev Stage
If your team hasn't started using dev stacks yet, the first step is to make a shared stack. This will be the start of your application pipeline.
A pipeline is a series of stages through which you can push code changes, with each stage gradually widening your user base. On large and long-running projects, pipelines can involve multiple stages and become fairly complicated. However, a simple pipeline consisting of just a dev and a prod stage is a solid start for teams just dipping their toes into the DevOps waters.
To create a dev stage, you first need to create a full application stack using a language such as AWS CloudFormation. Your stack should define everything that your application needs to run.
If you already have this for your production stack, then you're almost there! You may need to make a few adjustments based on how you want to launch your dev stack. You have a couple of choices here.
Launch in Same AWS Account as Prod
The simplest strategy is launching your dev stack in the same stack as your prod account. To do this, you'll need to parameterize your Infrastructure as Code deployment so that it uses different prefixes or suffixes for resource names. This will avoid naming collisions with your prod stack.
AWS CloudFormation makes this easy through the use of parameters. And actually, you don't even need to define your own parameters! You can use CloudFormation's pseudo-parameters - predefined metadata parameters - to implement this quickly and easily.
For example, assume you are defining an S3 bucket name and want to make sure it's distinct from your production bucket. Using CloudFormation, you can use the name of your stack as a prefix for the bucket name. In the example below (YAML), we use a regular CloudFormation stack parameter, AppVersion, and the full stack name as a pseudo-parameter to construct a unique name.
BucketName: !Sub "{$AWS:StackName}-${AppVersion}"
Launch in Separate AWS Account
However, it's not a great idea to mix stacks in a single account. Ideally, you want your production stack hosted in its own AWS account. This allows you to place additional restrictions on access to production. Such restrictions are almost a necessity if your team handles personally identifiable information on customers in prod.
If you launch your dev stack in a separate account, you don't need to worry about name conflicts. The only thing you should have to parameterize in this context are publicly facing values, such as your application's DNS endpoint.
A Dev Stack Per Developer
Creating a central dev stack is definitely a huge step forward. However, there's still room for improvement!
A central dev stack is fine for integrating changes that are getting close to production quality. Ideally, however, you want devs to be able to test in their own stacks before committing to a common Git branch. This reduces merge conflicts and helps ensure high-quality code early in the development process.
If you already have code for launching a dev stack, launching individual dev stacks for developers shouldn't involve much additional work. The major issue is tracking stacks and controlling costs. Giving your entire dev team unfettered access to an AWS account - even a non-production one - can leave you scrambling to control your cloud spend.
One approach is to use AWS Control Tower. Control Tower works in conjunction with AWS Organizations, which enables the creation and management of multiple AWS accounts under a single master account. You can use Control Tower in conjunction with AWS Service Catalog to offer your dev stack as a service catalog offering that developers can install into their accounts. You can even go one step farther and deploy the stack automatically as part of the account vending process.
Defining Your Branching Strategy with Dev Stacks
As I discussed in my last article, it's important when creating your CI/CD pipeline to work out a branching strategy. One of the simplest strategies is to use feature branches for development work. In feature branching, devs create a branch per feature. Developers use pull requests to request integration of their work into main.
Feature branching has several benefits. By using pull requests, other team members can review and vet a set of changes before they are integrated into the main branch. The entire process keeps your project's main branch clean and in a buildable, deployable state.
Whatever branching strategy you choose, there's little doubt that giving developers their own fully deployed stacks makes it easier to test changes before they ever hit main. The result is faster deployments and more reliable code.
TinyStacks Makes Dev Stacks Easy
Here at TinyStacks, we’re all about helping you deploy and manage your stacks in the cloud. We make it easy to transfer from a personal dev environment on your laptop into a development stage with a stack consistent with your production stack. Contact us today to find out more!