List ALL the availability zones!

Have you ever tried to google for a list of AWS Regions or Availability Zones? I do it all the time. I need to figure out which AZs to my ELB should go across, or want to try my new CloudFormation template in a different region to make sure I didn’t hardcode something I shouldn’t have. Of course, I usually blank out on how many zones each region has, and I always get my directions screwed up so I can never remember if Ireland is eu-west or eu-east (and while it is embarrassing to admit, I had no idea where São Paulo was before I started working with the AWS platform).

However, every time I google for a list of AZs, I remember that none of the AWS documentation gives a nice, easy list of all the regions and their availability zones. AWS does add new zones and regions pretty regularly, so a list might go out of date after a few months. Still, I end up looking for these every few days, so I wrote them all out, and then put it on the internet, so hopefully next time I google for “list of availability zones” this page pops up. Maybe it’ll help you too?

Virginia — US East 1
• us-east-1a
• us-east-1b
• us-east-1c
• us-east-1d
• us-east-1e

California — US West 1
• us-west-1a
• us-west-1b
• us-west-1c

Oregon — US West 2
• us-west-2a
• us-west-2b
• us-west-2c

Ireland — EU West 1
• eu-west-1a
• eu-west-1b
• eu-west-1c

Singapore — AP Southeast 1
• ap-southeast-1a
• ap-southeast-1b

Tokyo — AP Northeast 1
• ap-northeast-1a
• ap-northeast-1b
• ap-northeast-1c

Sydney — AP Southeast 2
• ap-southeast-2a
• ap-southeast-2b

São Paulo — SA East 1
• sa-east-1a
• sa-east-1b


Creating a Secure Deployment Pipeline in Amazon Web Services

Many organizations require a secure infrastructure. I’ve yet to meet a customer that says that security isn’t a concern. But, the decision on “how secure?” should be closely associated with a risk analysis for your organization.

Since Amazon Web Services (AWS) is often referred to as a “public cloud”, people sometimes infer that “public” must mean it’s “out in the public” for all to see. I’ve always seen “public/private clouds” as an unfortunate use of terms. In this context, public means more like “Public Utility”. People often interpret “private clouds” to be inherently more secure. Assuming that “public cloud” = less secure and “private cloud” = more secure couldn’t be further from the truth. Like most things, it’s all about how you architect your infrastructure. While you can define your infrastructure to have open access, AWS provides many tools to create a truly secure infrastructure while eliminating access to all but only authorized users.

I’ve created an initial list of many of the practices we use. We don’t employ all these practices in all situations, as it often depends on our customers’ particular security requirements. But, if someone asked me “How do I create a secure AWS infrastructure using a Deployment Pipeline?”, I’d offer some of these practices in the solution. I’ll be expanding these over the next few weeks, but I want to start with some of our practices.

AWS Security

* After initial AWS account creation and login, configure IAM so that there’s no need to use the AWS root account
* Apply least privilege to all IAM accounts. Be very careful about who gets Administrator access.
* Enable all IAM password rules
* Enable MFA for all users
* Secure all data at rest
* Secure all data in transit
* Put all AWS resources in a Virtual Private Cloud (VPC).
* No EC2 Key Pairs should be shared with others. Same goes for Access Keys.
* Only open required ports to the Internet. For example, with the exception of, say, port 80, no security groups should have a CIDR Source of The bastion host might have access to port 22 (SSH), but you should enable CIDR to limit access to specific subnets. Using a VPC is a part of a solution to eliminate Internet access. No canonical environments should have SSH/RDP access.
* Use IAM to limit access to specific AWS resources and/or remove/limit AWS console access
* Apply a bastion host configuration to reduce your attack profile
* Use IAM Roles so that there’s no need to configure Access Keys on the instances
* Use resource-level permissions in EC2 and RDS
* Use SSE to secure objects in S3 buckets
* Share initial IAM credentials with others through a secure mechanism (e.g. AES-256 encryption)
* Use and monitor AWS CloudTrail logs

Deployment Pipeline

A deployment pipeline is a staged process in which the complete software system is built and tested with every change. Team members receive feedback as it completes each stage. With most customers, we usually construct between 4-7 deployment pipeline stages and the pipeline only goes to the next stage if the previous stages were successful. If a stage fails, the whole pipeline instance fails. The first stage (often referred to as the “Commit Stage”) will usually take no more than 10 minutes to complete. Other stages may take longer than this. Most stages require no human intervention as the software system goes through more extensive testing on its way to production. With a deployment pipeline, software systems can be released at any time the business chooses to do so. Here are some of the security-based practices we employ in constructing a deployment pipeline.

* Automate everything: Networking (VPC, Route 53) Compute (EC2), Storage, etc. All AWS automation should be defined in CloudFormation. All environment configuration should be defined using infrastructure automation scripts – such as Chef, Puppet, etc.
* Version Everything: Application Code, Configuration, Infrastructure and Data
* Manage your binary dependencies. Be specific about binary version numbers. Ensure you have control over these binaries.
* Lockdown pipeline environments. Do not allow SSH/RDP access to any environment in the deployment pipeline
* For project that require it, use permissions on the CI server or Deployment application to limit who can run deployments in certain environments – such as QA, Pre-Production and Production. When you have a policy in which all changes are applied through automation and environments are locked down, this usually becomes less of a concern. But, it can still be a requirements on some teams.
* Use the Disposable Environments pattern – instances are terminated once every few days. This approach reduces the attack profile
* Log everything outside of the EC2 instances (so that they can be access later). Ensure these log files are encrypted e.g. securely through S3)
* All canonical changes are only applied through automation that are part of the deployment pipeline. This includes application, configuration, infrastructure and data change. Infrastructure patch management would be a part of the pipeline just like any outer software system change.
* No one has access to nor can make direct changes to pipeline environments
* Create high-availability systems Multi-AZ, Auto Scaling, Elastic Load Balancing and Route 53
* For non-Admin AWS users, only provide access to AWS through a secure Continuous Integration (CI) server or a self-service application
* Use Self-Service Deployments and give developers full SSH/RDP access to their self-service deployment. Only their particular EC2 Key Pair can access the instance(s) associated with the deployment. Self-Service Deployments can be defined in the CI server or a lightweight self-service application.
* Provide capability for any authorized user to perform a self-service deployment with full SSH/RDP access to the environment they created (while eliminating outside access)
* Run two active environments – We’ve yet to do this for customers, but if you want to eliminate all access to the canonical production environment, you might choose to run two active environments at once so that engineers can access the non-production environment to troubleshoot a problem in which the environment has the exact same configuration and data so you’re troubleshooting accurately.
* Run automated infrastructure tests to test for security vulnerabilities (e.g. cross-site scripting, SQL injections, etc.) with every change committed to the version-control repository as part of the deployment pipeline.


* What is a canonical environment? It’s your system of record. You want your canonical environment to be solely defined in source code and versioned. If someone makes a change to the canonical system and it affects everyone it should only be done through automation. While you can use a self-service deployment to get a copy of the canonical system, any direct change you make to the environment is isolated and never made part of the canonical system unless code is committed to the version-control repository.
* How can I troubleshoot if I cannot directly access canonical environments? Using a self-service deployment, you can usually determine the cause of the problem. If it’s a data-specific problem, you might import a copy of the production database. If this isn’t possible for time or security reasons, you might run multiple versions of the application at once.
* Why should we dispose of environments regularly? Two primary reasons. The first is to reduce your attack profile (i.e. if environments always go up and down, it’s more difficult to hone in on specific resources. The second reason is that it ensures that all team members are used to applying all canonical changes through automation and not relying on environments to always be up and running somewhere.
* Why should we lockdown environments? To prevent people from making disruptive environment changes that don’t go through the version-control repository.


How we use AWS OpsWorks

Amazon Web Services (AWS) OpsWorks was released one year ago this month. In the past year, we’ve used OpsWorks on several Cloud Delivery projects at Stelligent and at some of our customers. This article describes what’s worked for us and our customers. One of our core aims with any customer is to create a fully repeatable process for delivering software. To us, this translates into several more specific objectives. For each process we automate, the process must be fully documented, tested, scripted, versioned and continuous. This article describes how we achieved each of these five objectives in delivering OpsWorks solutions to our customers. In creating any solution, we version any and every asset required to create the software system. With the exception of certain binary packages, the entire software system gets described in code. This includes the application code, configuration, infrastructure and data.

As a note, we’ve developed other AWS solutions without OpsWorks using CloudFormation, Chef, Puppet and some of the other tools mentioned here, but the purpose of this is to describe our approach when using OpsWorks.

AWS Tools

AWS has over 30 services and we use a majority of these services when creating deployment pipelines for continuous delivery and automating infrastructure. However, we typically use only a few services directly when building these infrastructure. For instance, when creating infrastructure with OpsWorks, we’ll use the AWS Ruby SDK to provision the OpsWorks resources and CloudFormation for the resources we cannot provision through OpsWorks. We use these three services to access services such as EC2, Route 53, VPC, S3, Elastic Load Balancing, Auto Scaling, etc. These three services are described below.

AWS OpsWorks
– OpsWorks is an infrastructure orchestration and event modeling service for provisioning infrastructure resources. It also enables you to call out to Chef cookbooks (more on Chef later). The OpsWorks model logically defines infrastructure in terms of stacks, layers and apps. Within stacks, you can define layers; within layers you can define applications and within applications, you can run deployments. An event model automatically triggers events against these stacks (e.g. Setup, Configure, Deploy, Undeploy, Shutdown). As mentioned, we use the AWS API (through the Ruby SDK) to script the provisioning of all OpsWorks behavior. We never manually make changes to OpsWorks through the console (we make these changes to the versioned AWS API scripts).

CloudFormation – We use CloudFormation to automatically provision resources that we cannot provision directly through OpsWorks. For example, while OpsWorks connects with Virtual Private Clouds (VPC)s and Elastic Load Balancer (ELB)s, you cannot provision VPC or ELB directly through OpsWorks. Since we choose to script all infrastructure provisioning and workflow, we wrote CloudFormation templates for defining VPCs, ELBs, Relational Database Service (RDS) and Elasticache. We orchestrate the workflow in Jenkins so that these resources are automatically provisioned prior to provisioning the OpsWorks stacks. This way, the OpsWorks stacks can consume these resources that were provisioned in the CloudFormation templates. As with any other program, these templates are version-controlled.

AWS API (using Ruby SDK) – We use the AWS Ruby SDK to script the provisioning of OpsWorks stacks. While we avoid using the SDK directly for most other AWS services (because we can use CloudFormation), we chose to use the SDK for scripting OpsWorks because CloudFormation does not currently support OpsWorks. Everything that you might do using the OpsWorks dashboard – creating stacks, JSON configuration, calling out to Chef, deployments – are all written in Ruby programs that utilize the OpsWorks portion of the AWS API.

Infrastructure Automation

There are other non-AWS specific tools that we use in automating infrastructure. One of them is the infrastructure automation tool, Chef. Chef Solo is called from OpsWorks. We use infrastructure automation tools to script and as a way to document the process of provisioning infrastructure.

Chef – OpsWorks is designed to run Chef cookbooks (i.e. scripts/programs). Ultimately, Chef is where a bulk of the behavior for provisioning environments is defined – particularly once the EC2 instance is up and running. In Chef, we write recipes (logically stored in cookbooks) to install and configure web servers such as Apache and Nginx or application servers such as Rails and Tomcat. All of these Chef recipes are version-controlled and called from OpsWorks or CloudFormation.

Ubuntu – When using OpsWorks and there’s no specific operating system flavor requirement from our customer, we choose to use Ubuntu 12.04 LTS. We do this for two reasons. The first is that at the time of this writing, OpsWorks supports two Linux flavors: Amazon Linux and Ubuntu 12.04 LTS. The reason we choose Ubuntu is because it allows us to use Vagrant (more on Vagrant later). Vagrant provides us a way to test our Chef infrastructure automation scripts locally – increasing our infrastructure development speed.

Supporting Tools

Other supporting tools such as Jenkins, Vagrant and Cucumber help with Continuous Integration, local infrastructure development and testing. Each are described below.

JenkinsJenkins is a Continuous Integration server, but we also use it to orchestrate the coarse-grained workflow for the Cloud Delivery system and infrastructure for our customers. We use Jenkins fairly regularly in creating Cloud Delivery solutions for our customers. We configure Jenkins to run Cucumber features, build scripts, automated tests, static analysis, AWS Ruby SDK programs, CloudFormation templates and many more activities. Since Jenkins is an infrastructure component as well, we’ve automated the creation in OpsWorks and Chef and it also runs Cucumber features that we’ve written. These scripts and configuration are stored in Git as well and we can simply type a single command to get the Jenkins environment up and running. Any canonical changes to the Jenkins server are made by modifying the programs or configuration stored in Git.

VagrantVagrant runs a virtualized environment on your desktop and comes with support for certain OS flavors and environments. As mentioned, we use Vagrant to run and test our infrastructure automation scripts locally to increase the speed of development. In many cases, what might take 30-40 minutes to run the same Chef cookbooks can take 4-5 minutes to run locally in Vagrant – significantly increase our infrastructure development productivity.

Cucumber – We use Cucumber to write infrastructure specifications in code called features. This provides executable documented specifications that get run with each Jenkins build. Before we write any Chef, OpsWorks or CloudFormation code, we write Cucumber features. When completed, these features are run automatically after the Chef, OpsWorks and/or CloudFormation scripts provision the infrastructure to ensure the infrastructure is meeting the specifications described in the features. At first, these features are written without step definitions (i.e. they don’t actually verify behavior against the infrastructure), but then we iterate through a process of writing programs to automate the infrastructure provisioning while adding step definitions and refining the Cucumber features. Once all of this is hooked up to the Jenkins Continuous Integration server, it provisions the infrastructure and then runs the infrastructure tests/features written in Cucumber. Just like writing XUnit tests for the application code, this approach ensures our infrastructure behaves as designed and provides a set of regression tests that are run with every change to any part of the software system. So, Cucumber helps us document the feature as well as automate infrastructure tests. We also write usage and architecture documentation in READMEs, wikis, etc.


Continuous Delivery in the Cloud: Deployment Automation (Part 5 of 6)

In part 1 of this series, I introduced the Continuous Delivery (CD) pipeline for the Manatee Tracking application. In part 2 I went over how we use this CD pipeline to deliver software from checkin to production. In part 3, we focused on how CloudFormation is used to script the virtual AWS components that create the Manatee infrastructure. Then in part 4, we focused on a “property file less” environment by dynamically setting and retrieving properties. A list of topics for each of the articles is summarized below:
Part 1: Introduction – Introduction to continuous delivery in the cloud and the rest of the articles;
Part 2: CD Pipeline – In-depth look at the CD Pipeline;
Part 3: CloudFormation – Scripted virtual resource provisioning;
Part 4: Dynamic Configuration – “Property file less” infrastructure;
Part 5: Deployment Automation – What you’re reading now;
Part 6: Infrastructure Automation – Scripted environment provisioning (Infrastructure Automation)

In this part of the series, I am going to show how we use Capistrano to script our deployments to target environments.

What is Capistrano?
Capistrano is an open source Ruby tool used for deploying web applications. It automates deploying to one or more servers. These deployments can include procedures like placing a war on a target server, database changes, starting services, etc.

A Capistrano script has several major parts

  • Namespaces: Namespaces in Capistrano are used for differentiating tasks from other tasks with the same name. This is important if you create a library out of your Capistrano deployment configuration, you will want to make sure your tasks are unique. For instance a typical name for a task is setup. You need to make sure that your setup task does not potentially interfere with another user’s custom setup task. By using namespaces, you won’t have this conflict.
  • Tasks: Tasks are used for performing specific operations. An example task would be setup. Inside the setup task you will generally prepare the server for subsequent steps to execute successfully like deleting the current application.
  • Variables: Variables in Capistrano are defined as ruby symbols. These are set initially and then referenced later on in the script.
  • Order of execution: Capistrano allows you to define the order of deployment execution. You do this with Capistrano’s built in feature after. With after you define the order of task execution during your Capistrano deployment.
  • Templates: Templates are files that have injected ruby snippets. These are used for dynamically building configuration files.
  • Roles: Roles define what part each server in your infrastructure plays in the deployment. Typical roles consist of db, web and app. Roles are then referenced inside your tasks to determine which server the task is run against.

Since Capistrano is a ruby based tool, you can inject ruby methods and operations to enhance Capistrano. In our deployment we use ruby for returning property values from SimpleDB – as we discussed in part 4 of this series, Dynamic Configuration. This enables us to dynamically deploy to target servers.

How do you install Capistrano?

1. Capistrano is available as a rubygem. You simply type gem install capistrano on your Linux machine (assuming you have ruby and rubygems installed)
2. Type capify . this will create a Capfile which is a main file that Capistrano needs and a config/deploy.rb file (which is your actual Capistrano deployment script).

How do you run a Capistrano script?
You run Capistrano from the command line. From the same directory as your Capfile, type cap namespace:task. namespace and task being your own Namespace and Task defined in your deploy.rb script. This will start your Capistrano deployment.

Why do we use Capistrano?
We use Capistrano in order to have a fully scripted, versioned deployment. Every step in our application deployment is scripted and fully automated – which reduces errors when deploying. This gives us complete control over our deployment and the ability to deploy whenever we are ready.

Capistrano for Manatees
In the Manatee deployment, we use Capistrano for deploying our Manatee tracking application to our target environment. I am going to go through each part of the deploy.rb and explain its use and purpose. In a deployment’s lifecycle, the deployment is run as part of the CD pipeline – discussed in part 2 of the series, CD Pipeline. I’ll first go through a high level summary of the deployment and then dive into more detail in the next section.

1. Variables are set, which includes returning several properties from SimpleDB
2. Roles are set: db, web and app are all set to the ip_address variable configured dynamically in Step #1
3. The order of execution is set to run the tasks in order
4. Tasks are executed
5. If all tasks are executed successfully, Capistrano signals deployment success.

Now that we know at a high level what’s being done during the deployment, lets take a deeper look at the inside of the script. The actual script can be found here: deploy.rb


Command line set
stack – Passed into SimpleDB to return the dynamically set property values
ssh_key – Used by the ssh_options variable to SSH into the target environment

Dynamically set
domain – Used by the application variable
artifact_bucket - Used to build the artifact_url variable
ip_address – Used to define the IP address of the target environment to SSH into
dataSourceUsername – Returns a value that is part of the wildtracks_config.properties file
dataSourcePassword – Returns a value that is part of the wildtracks_config.properties file
dataStorageFtpUsername – Returns a value that is part of the wildtracks_config.properties file
dataStorageFtpPassword – Returns a value that is part of the wildtracks_config.properties file

user – The user to SSH into the target box as
use_sudo – Define whether to prepend every command with sudo or not
deploy_to – Defines the deployment directory on the target environment
artifact – The artifact to deploy to the target server
artifact_url – The URL for downloading the artifact
ssh_options – Specialized SSH configuration
application –  Used to set the domain that application runs on
liquibase_jar – Location of the liquibase.jar on the deployment server
postgres_jar – Location of the postgres.jar on the deployment server


Since the app server, web server, and database all co exist on the same environment, we set each of these to the same variable, ip_address.


Deploy: We use deploy as our namespace. Since we aren’t distributing this set of deployment tasks, we don’t need to make a unique namespace. In fact we could remove the namespace all-together, but we wanted to show it being used.

namespace :deploy


We define our execution order at the bottom of the script using after. This coordinates which task should be run during the deployment.

after "deploy:setup", "deploy:wildtracks_config"
after "deploy:wildtracks_config", "deploy:httpd_conf"
after "deploy:httpd_conf", "deploy:deploy"
after "deploy:deploy", "deploy:liquibase"
after "deploy:deploy", "deploy:restart"


  • Setup: The setup task is our initial task. It makes sure the ownership of our deployment directory is set of tomcat. It then stops httpd and tomcat to get ready for the deployment.

    task :setup do
      run "sudo chown -R tomcat:tomcat #{deploy_to}"
      run "sudo service httpd stop"
      run "sudo service tomcat6 stop"
  • wildtracks_config: The wildtracks_config task is the second task to run. It dynamically creates the wildtracks-config.properties file using a template and the variables set previously in the script. It then places the wildtracks-config.properties file on the target environment.

    task :wildtracks_config, :roles => :app do

      set :dataSourceUsername do
        item = sdb.domains["stacks"].items["wildtracks-config"]
      set :dataSourcePassword do
        item = sdb.domains["stacks"].items["wildtracks-config"]
      set :dataStorageFtpUsername do
        item = sdb.domains["stacks"].items["wildtracks-config"]
      set :dataStorageFtpPassword do
        item = sdb.domains["stacks"].items["wildtracks-config"]

      set :dataSourceUrl, "jdbc:postgresql://localhost:5432/manatees_wildtrack"
      set :dataStorageWorkDir, "/var/tmp/manatees_wildtracks_workdir"
      set :dataStorageFtpUrl, "ftp.wildtracks.org"
      set :databaseBackupScriptFile, "/usr/share/tomcat6/.sarvatix/manatees/wildtracks/database_backups/script/db_backup.sh"

      config_content = from_template("config/templates/wildtracks-config.properties.erb")
      put config_content, "/home/ec2-user/wildtracks-config.properties"

      run "sudo mv /home/ec2-user/wildtracks-config.properties /usr/share/tomcat6/.sarvatix/manatees/wildtracks/wildtracks-config.properties"
      run "sudo chown -R tomcat:tomcat /usr/share/tomcat6/.sarvatix/manatees/wildtracks/wildtracks-config.properties"
      run "sudo chmod 777 /usr/share/tomcat6/.sarvatix/manatees/wildtracks/wildtracks-config.properties"

  • httpd_conf: The httpd_conf task is third on the stack and performs a similar function to the wildtracks_config task, but with the httpd.conf configuration file.

    task :httpd_conf, :roles => :app do

      config_content = from_template("config/templates/httpd.conf.erb")
      put config_content, "/home/ec2-user/httpd.conf"

      run "sudo mv /home/ec2-user/httpd.conf /etc/httpd/conf/httpd.conf"

  • Deploy: The deploy task is where the actual deployment of the application code is done. This task removes the current version of the application and downloads the latest.

    task :deploy do
      run "cd #{deploy_to} && sudo rm -rf wildtracks* && sudo wget #{artifact_url}"
  • Liquibase: The liquibase task sets up and ensures that the database is configured correctly.

    task :liquibase, :roles => :db do

      db_username = fetch(:dataSourceUsername)
      db_password = fetch(:dataSourcePassword)
      private_ip_address = fetch(:private_ip_address)

      set :liquibase_jar, "/usr/share/tomcat6/.grails/1.3.7/projects/Build/plugins/liquibase-"
      set :postgres_jar, "/usr/share/tomcat6/.ivy2/cache/postgresql/postgresql/jars/postgresql-8.4-701.jdbc3.jar"

      system("cp -rf /usr/share/tomcat6/.jenkins/workspace/DeployManateeApplication/grails-app/migrations/* /usr/share/tomcat6/.jenkins/workspace/DeployManateeApplication/")

      system("java -jar #{liquibase_jar}\

  • Restart: Lastly the restart task starts the httpd and tomcat services.

    task :restart, :roles => :app do
      run "sudo service httpd restart"
      run "sudo service tomcat6 restart"

Now that we’ve gone through the deployment, we need to test it. For testing our deployments, we use Cucumber. Cucumber enables us to do acceptance testing on our deployment. We verify that the application is up and available, the correct services are started and the property files are stored in the right locations.

Capistrano allows us to completely script and version our deployments enabling our deployments to be run at anytime. With Capistrano’s automation in conjunction with Cucumber’s acceptance testing, we are given a high level of confidence in our deployments and that when they are run, the application will be deployed successfully.

In the next and last part of our series – Infrastructure Automation – we’ll go through scripting environment using an industry standard infrastructure automation tool, Puppet.


Continuous Delivery in the Cloud: Dynamic Configuration (Part 4 of 6)

In part 1 of this series, I introduced the Continuous Delivery (CD) pipeline for the Manatee Tracking application. In part 2 I went over how we use this CD pipeline to deliver software from checkin to production. In part 3, we focused on how CloudFormation is used to script the virtual AWS components that create the Manatee infrastructure. A list of topics for each of the articles is summarized below:

Part 1: Introduction – Introduction to continuous delivery in the cloud and the rest of the articles;
Part 2: CD Pipeline – In-depth look at the CD Pipeline;
Part 3: CloudFormation – Scripted virtual resource provisioning;
Part 4: Dynamic Configuration –  What you’re reading now;
Part 5: Deployment Automation – Scripted deployment orchestration;
Part 6: Infrastructure Automation – Scripted environment provisioning (Infrastructure Automation)

In this part of the series, I am going to explain how we dynamically generate our configuration and avoid property files whenever possible. Instead of using property files, we store and retrieve configuration on the fly – as part of the CD pipeline – without predefining these values in a static file (i.e. a properties file) ahead of time. We do this using two methods: AWS SimpleDB and CloudFormation.

SimpleDB is a highly available non-relational data storage service that only stores strings in key value pairs. CloudFormation, as discussed in Part 3 of the series, is a scripting language for allocating and configuring AWS virtual resources.

Using SimpleDB

Throughout the CD pipeline, we often need to manage state across multiple Jenkins jobs. To do this, we use SimpleDB. As the pipeline executes, values that will be needed by subsequent jobs get stored in SimpleDB as properties. When the properties are needed we use a simple Ruby script script to return the key/value pair from SimpleDB and then use it as part of the job. The values being stored and retrieved range from IP addresses and domain names to AMI (Machine Images) IDs.

So what makes this dynamic? As Jenkins jobs or CloudFormation templates are run, we often end up with properties that need to be used elsewhere. Instead of hard coding all of the values to be used in a property file, we create, store and retrieve them as the pipeline executes.

Below is the CreateTargetEnvironment Jenkins job script that creates a new target environment from a CloudFormation script production.template

if [ $deployToProduction ] == true

# Create Cloudformaton Stack
ruby /usr/share/tomcat6/scripts/aws/create_stack.rb ${STACK_NAME} ${WORKSPACE}/production.template ${HOST} ${JENKINSIP} ${SSH_KEY} ${SGID} ${SNS_TOPIC}

# Load SimpleDB Domain with Key/Value Pairs
ruby /usr/share/tomcat6/scripts/aws/load_domain.rb ${STACK_NAME}

# Pull and store variables from SimpleDB
host=`ruby /usr/share/tomcat6/scripts/aws/showback_domain.rb ${STACK_NAME} InstanceIPAddress`

# Run Acceptance Tests
cucumber features/production.feature host=${host} user=ec2-user key=/usr/share/tomcat6/.ssh/id_rsa

Referenced above in the CreateTargetEnvironment code snippet. This is the load_domain.rb script that iterates over a file and sends key/value pairs to SimpleDB.

require 'rubygems'
require 'aws-sdk'
load File.expand_path('../../config/aws.config', __FILE__)


file = File.open("/tmp/properties", "r")

sdb = AWS::SimpleDB.new

AWS::SimpleDB.consistent_reads do
  domain = sdb.domains["stacks"]
  item = domain.items["#{stackname}"]

  file.each_line do|line|
    key,value = line.split '='
      "#{key}" => "#{value}")

Referenced above in the CreateTargetEnvironment code snippet. This is the showback_domain.rb script which connects to SimpleDB and returns a key/value pair.

load File.expand_path('../../config/aws.config', __FILE__)


sdb = AWS::SimpleDB.new

AWS::SimpleDB.consistent_reads do
  domain = sdb.domains["stacks"]
  item = domain.items["#{item_name}"]

  item.attributes.each_value do |name, value|
    if name == "#{key}"
      puts "#{value}".chomp

In the above in the CreateTargetEnvironment code snippet, we store the outputs of the CloudFormation stack in a temporary file. We then iterate over the file with the load_domain.rb script and store the key/value pairs in SimpleDB.

Following this, we make a call to SimpleDB with the showback_domain.rb script and return the instance IP address (created in the CloudFormation template) and store it in the host variable. host is then used by cucumber to ssh into the target instance and run the acceptance tests.

Using CloudFormation

In our CloudFormation templates we allocate multiple AWS resources. Every time we run the template, a different resource is being used. For example, in our jenkins.template we create a new IAM user. Every time we run the template a different IAM user with different credentials is created. We need a way to reference these resources. This is where CloudFormation comes in. You can reference resources within other resources throughout the script. You can define a reference to another resource using the Ref function in CloudFormation. Using Ref, you can dynamically refer to values of other resources such as an IP Address, domain name, etc.

In the script we are creating an IAM user, referencing the IAM user to create AWS Access keys and then storing them in an environment variable.

"CfnUser" : {
  "Type" : "AWS::IAM::User",
  "Properties" : {
    "Path": "/",
    "Policies": [{
      "PolicyName": "root",
      "PolicyDocument": {

"HostKeys" : {
  "Type" : "AWS::IAM::AccessKey",
  "Properties" : {
    "UserName" : { "Ref": "CfnUser" }

"# Add AWS Credentials to Tomcat\n",
"echo \"AWS_ACCESS_KEY=", { "Ref" : "HostKeys" }, "\" >> /etc/sysconfig/tomcat6\n",
"echo \"AWS_SECRET_ACCESS_KEY=", {"Fn::GetAtt": ["HostKeys", "SecretAccessKey"]}, "\" >> /etc/sysconfig/tomcat6\n",

We can then use these access keys in other scripts by referencing the $AWS_ACCESS_KEY and $AWS_SECRET_ACCESS_KEY environment variables.

How is this different from typical configuration management?

Typically in many organizations, there’s a big property with hard coded key/value pairs that gets passed into the pipeline. The pipeline executes using the given parameters and cannot scale or change without a user modifying the property file. They are unable to scale or adapt because all of the properties are hard coded, if the property file hard codes the IP to an EC2 instance and it goes down for whatever reason, their pipeline doesn’t work until someone fixes the property file. There are more effective ways of doing this when using the cloud. The cloud is provides on-demand resources that will constantly be changing. These resources will have different IP addresses, domain names, etc associated with them every time.

With dynamic configuration, there are no property files, every property is generated as part of the pipeline.

With this dynamic approach, the pipeline values change with every run. As new cloud resources are allocated, the pipeline is able to adjust itself and automatically without the need for users to constantly modify property files. This leads to less time spent debugging those cumbersome property file management issues that plague most companies.

In the next part of our series – which is all about Deployment Automation – we’ll go through scripting and testing your deployment using industry-standard tools. In this next article, you’ll see how to orchestrate deployment sequences and configuration using Capistrano.


Continuous Delivery in the Cloud: CD Pipeline (Part 2 of 6)

In part 1 of this series, I introduced the Continuous Delivery (CD) pipeline for the Manatee Tracking application and how we use this pipeline to deliver software from checkin to production. In this article I will take an in-depth look at the CD pipeline. A list of topics for each of the articles is summarized below.

Part 1: Introduction – Introduction to continuous delivery in the cloud and the rest of the articles;
Part 2: CD Pipeline – What you’re reading now;
Part 3: CloudFormation – Scripted virtual resource provisioning;
Part 4: Dynamic Configuration – “Property file less” infrastructure;
Part 5: Deployment Automation – Scripted deployment orchestration;
Part 6: Infrastructure Automation – Scripted environment provisioning (Infrastructure Automation)

The CD pipeline consists of five Jenkins jobs. These jobs are configured to run one after the other. If any one of the jobs fail, the pipeline fails and that release candidate cannot be released to production. The five Jenkins jobs are listed below (further details of these jobs are provided later in the article).

  1. 1) A job that set the variables used throughout the pipeline (SetupVariables)
  2. 2) Build job (Build)
  3. 3) Production database update job (StoreLatestProductionData)
  4. 4) Target environment creation job (CreateTargetEnvironment)
  5. 5) A deployment job (DeployManateeApplication) which enables a one-click deployment into production.

We used Jenkins plugins to add additional features to the core Jenkins configuration. You can extend the standard Jenkins setup by using Jenkins plugins. A list of the plugins we use for the Sea to Shore Alliance Continuous Delivery configuration are listed below.

Grails: http://updates.jenkins-ci.org/download/plugins/grails/1.5/grails.hpi
Groovy: http://updates.jenkins-ci.org/download/plugins/groovy/1.12/groovy.hpi
Subversion: http://updates.jenkins-ci.org/download/plugins/subversion/1.40/subversion.hpi
Paramterized Trigger: http://updates.jenkins-ci.org/download/plugins/parameterized-trigger/2.15/parameterized-trigger.hpi
Copy Artifact: http://updates.jenkins-ci.org/download/plugins/copyartifact/1.21/copyartifact.hpi
Build Pipeline: http://updates.jenkins-ci.org/download/plugins/build-pipeline-plugin/1.2.3/build-pipeline-plugin.hpi
Ant: http://updates.jenkins-ci.org/download/plugins/ant/1.1/ant.hpi
S3: http://updates.jenkins-ci.org/download/plugins/s3/0.2.0/s3.hpi

The parameterized trigger, build pipeline and S3 plugins are used for moving the application through the pipeline jobs. The Ant, Groovy, and Grails plugins are used for running the build for the application code. Subversion for polling and checking out from version control.

Below, I describe each of the jobs that make up the CD pipeline in greater detail.

SetupVariables: Jenkins job used for entering in necessary property values which are propagated along the rest of the pipeline.

Parameter: STACK_NAME
Type: String
Where: Used in both CreateTargetEnvironment and DeployManateeApplication jobs
Purpose: Defines the CloudFormation Stack name and SimpleDB property domain associated with the CloudFormation stack.

Parameter: HOST
Type: String
Where: Used in both CreateTargetEnvironment and DeployManateeApplication jobs
Purpose: Defines the CNAME of the domain created in the CreateTargetEnvironment job. The DeployManateeApplication job uses it when it dynamically creates configuration files. For instance, in test.oneclickdeployment.com, test would be the HOST

Type: String
Where: Used in the StoreProductionData job
Purpose: Sets the production IP for the job so that it can SSH into the existing production environment and run a database script that exports the data and uploads it to S3.

Parameter: deployToProduction
Type: Boolean
Where: Used in both CreateTargetEnvironment and DeployManateeApplication jobs
Purpose: Determines whether to use the development or production SSH keypair.

In order for the parameters to propagate through the pipeline, we pass the current build parameters using the parametrized build trigger plugin

Build: Compiles the Manatee application’s Grails source code and creates a WAR file.

To do this, we utilize a Jenkins grails plugin and run grails targets such as compile and prod war. Next, we archive the grails migrations for use in the DeployManateeApplication job and then the job pushes the Manatee WAR up to S3 which is used as an artifact repository.

Lastly, using the trigger parametrized build plugin, we trigger the StoreProductionData job with the current build parameters.

StoreProductionData: This job performs a pg dump (PostgreSQL dump) of the production database and then stores it up in S3 for the environment creation job to use when building up the environment. Below is a snippet from this job.

ssh -i /usr/share/tomcat6/development.pem -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no ec2-user@${PRODUCTION_IP} ruby /home/ec2-user/database_update.rb

On the target environments created using the CD pipeline, a database script is stored. The script goes into the PostgreSQL database and runs a pg_dump. It then pushes the pg_dump SQL file to S3 to be used when creating the target environment.

After the SQL file is stored successfully, the CreateTargetEnvironment job is triggered.

CreateTargetEnvironment: Creates a new target environment using a CloudFormation template to create all the AWS resources and calls puppet to provision the environment itself from a base operating system to a fully working target environment ready for deployment. Below is a snippet from this job.

if [ $deployToProduction ]

# Create Cloudformaton Stack
ruby ${WORKSPACE}/config/aws/create_stack.rb ${STACK_NAME} ${WORKSPACE}/infrastructure/manatees/production.template ${HOST} ${JENKINSIP} ${SSH_KEY} ${SGID} ${SNS_TOPIC}

# Load SimpleDB Domain with Key/Value Pairs
ruby ${WORKSPACE}/config/aws/load_domain.rb ${STACK_NAME}

# Pull and store variables from SimpleDB
host=`ruby ${WORKSPACE}/config/aws/showback_domain.rb ${STACK_NAME} InstanceIPAddress`

# Run Acceptance Tests
cucumber ${WORKSPACE}/infrastructure/manatees/features/production.feature host=${host} user=ec2-user key=/usr/share/tomcat6/.ssh/id_rsa

# Publish notifications to SNS
sns-publish --topic-arn $SNS_TOPIC --subject "New Environment Ready" --message "Your new environment is ready. IP Address: $host. An example command to ssh into the box would be: ssh -i development.pem ec2-user@$host This instance was created by $JENKINS_DOMAIN" --aws-credential-file /usr/share/tomcat6/aws_access

Once the environment is created, a set of Cucumber tests is run to ensure it’s in the correct working state. If any test fails, the entire pipeline fails and the developer is notified something went wrong. Otherwise if it passes, the DeployManateeApplication job is kicked off and an AWS SNS email notification with information to access the new instance is sent to the developer.

DeployManateeApplication: Runs a Capistrano script which uses steps in order to coordinate the deployment. A snippet from this job is displayed below.

if [ !$deployToProduction ]


cap deploy:setup stack=${STACK_NAME} key=${SSH_KEY}

sed -i "s@manatee0@${HOST}@" ${WORKSPACE}/deployment/features/deployment.feature

host=`ruby ${WORKSPACE}/config/aws/showback_domain.rb ${STACK_NAME} InstanceIPAddress`
cucumber deployment/features/deployment.feature host=${host} user=ec2-user key=${SSH_KEY} artifact=

sns-publish --topic-arn $SNS_TOPIC --subject "Manatee Application Deployed" --message "Your Manatee Application has been deployed successfully. You can view it by going to http://$host/wildtracks This instance was deployed to by $JENKINS_DOMAIN" --aws-credential-file /usr/share/tomcat6/aws_access

This deployment job is the final piece of the delivery pipeline, it pulls together all of the pieces created in the previous jobs to successfully deliver working software.

During the deployment, the Capistrano script SSH’s into the target server, deploys the new war and updated configuration changes and restarts all services. Then the Cucumber tests are run to ensure the application is available and running successfully. Assuming the tests pass, an AWS SNS email gets dispatched to the developer with information on how to access their new development application

We use Jenkins as the orchestrator of the pipeline. Jenkins executes a set of scripts and passes around parameters as it runs each job. Because of the role Jenkins plays, we want to make sure it’s treated the same way as application – meaning versioning and testing all of our changes to the system. For example, if a developer modifies the create environment job configuration, we want to have the ability to revert back if necessary. Due to this requirement we version the Jenkins configuration. The jobs, plugins and main configuration. To do this, a script is executed each hour using cron.hourly that checks for new jobs or updated configuration and commits them up to version control.

The CD pipeline that we have built for the Manatee application enables any change in the application, infrastructure, database or configuration to move through to production seamlessly using automation. This allows any new features, security fixes, etc. to be fully tested as it gets delivered to production at the click of a button.

In the next part of our series – which is all about using CloudFormation – we’ll go through a CloudFormation template used to automate the creation of a Jenkins environment. In this next article, you’ll see how CloudFormation procures AWS resources and provisions our Jenkins CD Pipeline environment.

Continuous Delivery in the Cloud Case Study for the Sea to Shore Alliance – Introduction (part 1 of 6)

We help companies deliver software reliably and repeatedly using Continuous Delivery in the Cloud. With Continuous Delivery (CD), teams can deliver new versions of software to production by flattening the software delivery process and decreasing the cycle time between an idea and usable software through the automation of the entire delivery system: build, deployment, test, and release. CD is enabled through a delivery pipeline. With CD, our customers can choose when and how often to release to production. On top of this, we utilize the cloud so that customers can scale their infrastructure up and down and deliver software to users on demand.

Stelligent offers a solution called Elastic Operations which provides a Continuous Delivery platform along with expert engineering support and monitoring of a delivery pipeline that builds, tests, provisions and deploys software to target environments – as often as our customers choose. We’re in the process of open sourcing the platform utilized by Elastic Operations.

In this six-part blog series, I am going to go over how we built out a Continuous Delivery solution for the Sea to Shore Alliance:

Part 1: Introduction – What you’re reading now;
Part 2: CD Pipeline – Automated pipeline to build, test, deploy, and release software continuously;
Part 3: CloudFormation – Scripted virtual resource provisioning;
Part 4: Dynamic Configuration – “Property file less” infrastructure;
Part 5: Deployment Automation – Scripted deployment orchestration;
Part 6: Infrastructure Automation – Scripted environment provisioning (Infrastructure Automation)

This year, we delivered this Continuous Delivery in the Cloud solution to the Sea to Shore Alliance. The Sea to Shore Alliance is a non-profit organization whose mission is to protect and conserve the world’s fragile coastal ecosystems and its endangered species such as manatees, sea turtles, and right whales. One of their first software systems tracks and monitors manatees. Prior to Stelligent‘s involvement, the application was running on a single instance that was manually provisioned and deployed. As a result of the manual processes, there were no automated tests for the infrastructure or deployment. This made it impossible to reproduce environments or deployments the same way every time. Moreover, the knowledge to recreate these environments, builds and deployments were locked in the heads of a few key individuals. The production application for tracking these Manatees, developed by Sarvatix, is located here.

In this case study, I describe how we went from an untested manual process in which the development team was manually building software artifacts, creating environments and deploying, to a completely automated delivery pipeline that is triggered with every change.

Figure 1 illustrates the AWS architecture of the infrastructure that we designed for this Continuous Delivery solution.

There are two CloudFormation stacks being used, the Jenkins stack – or Jenkins environment – as shown on the left and the Manatee stack – or Target environment – as shown on the right.

The Jenkins Stack

  1. * Creates the jenkins.example.com Route53 Hosted Zone
  2. * Creates an EC2 instance with Tomcat and Jenkins installed and configured on it.
  3. * Runs the CD Pipeline

The Manatee stack is slightly different, it utilizes the configuration provided by SimpleDB to create itself. This stack defines the target environment for which the application software is deployed.

The Manatee Stack

  1. * Creates the manatee.example.com Route53 Hosted Zone
  2. * Creates an EC2 instance with Tomcat, Apache, PostgreSQL installed on it.
  3. * Runs the Manatee application.

The Manatee stack is configured with CPU alarms that send an email notification to the developers/administrators when it becomes over-utilized. We’re in the process of scaling to additional instances when these types of alarms are triggered.

Both instances are encapsulated behind a security group so that they can talk between each other using the internal AWS network.

Fast Facts
Industry: Non-Profit
Profile: Customer tracks and monitors endangered species such as manatees.
Key Business Issues: The customer’s development team needed to have unencumbered access to resources along with automated environment creation and deployment.
Stakeholders: Development team and scientists and others from the Sea to Shore Alliance
Solution: Continuous Delivery in the Cloud (Elastic Operations)
Key Tools/Technologies: AWS – Amazon Web Services (CloudFormation, EC2, S3, SimpleDB, IAM, CloudWatch, SNS), Jenkins, Capistrano, Puppet, Subversion, Cucumber, Liquibase

The Business Problem
The customer needed an operations team that could be scaled up or down depending on the application need. The customer’s main requirements were to have unencumbered access to resources such as virtual hardware. Specifically, they wanted to have the ability to create a target environment and run an automated deployment to it without going to a separate team and submitting tickets, emails, etc. In addition to being able to create environments, the customer wanted to have more control over the resources being used; they wanted to have the ability to terminate resources if they were unused. To address these requirements we introduced an entirely automated solution which utilizes the AWS cloud for providing resources on-demand, along with other solutions for providing testing, environment provisioning and deployment.

On the Manatee project, we have five key objectives for the delivery infrastructure. The development team should be able to:

  1. * Deliver new software or updates to users on demand
  2. * Reprovision target environment configuration on demand
  3. * Provision environments on demand
  4. * Remove configuration bottlenecks
  5. * Ability for users to terminate instances

Our Team
Stelligent’s team consisted of an account manager and one polyskilled DevOps Engineer that built, managed, and supported the Continuous Delivery pipeline.

Our Solution
Our solution, a single delivery pipeline that gives our customer (developers, testers, etc.) unencumbered access to resources and a single click automated deployment to production. To enable this, the pipeline needed to include:

  1. * The ability for any authorized team member to create a new target environment using a single click
  2. * Automated deployment to the target environment
  3. * End-to-end testing
  4. * The ability to terminate unnecessary environments
  5. * Automated deployment into production with a single click

The delivery pipeline improves efficiency and reduces costs by not limiting the development team. The solution includes:

  • On-Demand Provisioning – All hardware is provided via EC2’s virtual instances in the cloud, on demand. As part of the CD pipeline, any authorized team member can use the Jenkins CreateTargetEnvironment job to order target environments for development work.
  • Continuous Delivery Solution so that the team can deliver software to users on demand:
  • Development Infrastructure – Consists of:
    • Tomcat: used for hosting the Manatee Application
    • Apache: Hosted the front-end website and used virtual hosts for proxying and redirection.
    • PostgreSQL: Database for the Manatee application
    • Groovy: the application is written in Grails which uses Groovy.
  • Instance Management – Any authorized team member is able to monitor virtual instance usage by viewing Jenkins. There is a policy that test instances are automatically terminated every two days. This promotes ephemeral environments and test automation.
  • Deployment to Production – There’s a boolean value (i.e. a checkbox the user selects) in the delivery pipeline used for deciding whether to deploy to production.
  • System Monitoring and Disaster Recovery – Using the AWS CloudWatch service, AWS provides us with detailed monitoring to notify us of instance errors or anomalies through statistics such as CPU utilization, Network IO, Disk utilization, etc. Using these solutions we’ve implemented an automated disaster recovery solution.

A list of the AWS tools we utilized are enumerated below.

Tool: AWS EC2
What is it? Cloud-based virtual hardware instances
Our Use: We use EC2 for all of our virtual hardware needs. All instances, from development to production are run on EC2

Tool: AWS S3
What is it? Cloud-based storage
Our Use: We use S3 as both a binary repository and a place to store successful build artifacts.

Tool:  AWS IAM
What is it? User-based access to AWS resources
Our Use: We create users dynamically and use their AWS access and secret access keys so we don’t have to store credentials as properties

Tool: AWS CloudWatch
What is it? System monitoring
Our Use: Monitors all instances in production. If an instance takes an abnormal amount of strain or shuts down unexpectedly, SNS sends an email to designated parties

What is it? Email notifications
Our Use: When an environment is created or a deployment is run, SNS is used to send notifications to affected parties.

Tool: Cucumber
What is it? Acceptance testing
Our Use: Cucumber is used for testing at almost every step of the way. We use Cucumber to test infrastructure, deployments and application code to ensure correct functionality. Cucumber’s unique english-ess  verbiage allows both technical personnel and customers to communicate using an executable test.

Tool: Liquibase
What is it? Automated database change management
Our Use: Liquibase is used for all database changesets. When a change is necessary within the database, it is made to a liquibase changelog.xml

Tool: AWS CloudFormation
What is it? Templating language for orchestrating all AWS resources
Our Use: CloudFormation is used for creating a fully working Jenkins environment and Target environment. For instance for the Jenkins environment it creates the EC2 instance with CloudWatch monitoring alarms, associated IAM user, SNS notification topic, everything required for Jenkins to build. This along with Jenkins are the major pieces of the infrastructure.

Tool: AWS SimpleDB
What is it? Cloud-based NoSQL database
Our Use: SimpleDB is used for storing dynamic property configuration and passing properties through the CD Pipeline. As part of the environment creation process, we store multiple values such as IP addresses that we need when deploying the application to the created environment.

Tool: Jenkins
What is it? We’re using Jenkins to implement a CD pipeline using the Build Pipeline plugin.
Our Use: Jenkins runs the CD pipeline which does the building, testing, environment creation and deploying. Since the CD pipeline is also code (i.e. configuration code), we version our Jenkins configuration.

Tool: Capistrano
What is it? Deployment automation
Our Use: Capistrano orchestrates and automates deployments. Capistrano is a Ruby-based deployment DSL that can be used to deploy to multiple platforms including Java, Ruby and PHP. It is called as part of the CD pipeline and deploys to the target environment.

Tool: Puppet
What is it? Infrastructure automation
Our Use: Puppet takes care of the environment provisioning. CloudFormation requests the environment and then calls Puppet to do the dynamic configuration. We configured Puppet to install, configure, and manage the packages, files and services.

Tool: Subversion
What is it? Version control system
Our Use: Subversion is the version control repository where every piece of the Manatee infrastructure is stored. This includes the environment scripts such as the Puppet modules, the CloudFormation templates, Capistrano deployment scripts, etc.

We applied the on-demand usability of the cloud with a proven continuous delivery approach to build an automated one click method for building and deploying software into scripted production environments.

In the blog series, I will describe the technical implementation of how we went about building this infrastructure into a complete solution for continuously delivering software. This series will consist of the following:

Part 2 of 6 – CD Pipeline: I will go through the technical implementation of the CD pipeline using Jenkins. I will also cover Jenkins versioning, pulling and pushing artifacts from S3, and Continuous Integration.

Part 3 of 6 – CloudFormation: I will go through a CloudFormation template we’re using to orchestrate the creation of AWS resources and to build the Jenkins and target infrastructure.

Part 4 of 6 – Dynamic Configuration: Will cover dynamic property configuration using SimpleDB

Part 5 of 6 – Deployment Automation: I will explain Capistrano in detail along how we used Capistrano to deploy build artifacts and run Liquibase database changesets against target environments

Part 6 of 6 – Infrastructure Automation: I will describe the features of Puppet in detail along with how we’re using Puppet to build and configure target environments – for which the software is deployed.


DevOps in the Cloud LiveLessons (Video)

DevOps in the Cloud LiveLessons walks viewers through the process of putting together a complete continuous delivery platform for a working software application written in Ruby on Rails along with examples in other development platforms such as Grails and Java on the companion website. These applications are deployed to Amazon Web Services (AWS), which is an infrastructure as a service, commonly referred to as “the cloud”. Paul M. Duvall goes through the pieces that make up this platform including infrastructure and environments, continuous integration, build and deployment scripting, testing and database. Also, viewers will learn configuration management and collaboration practices and techniques along with what those nascent terms known as DevOps, continuous delivery and continuous deployment are all about. Finally, since this LiveLesson focuses on deploying to the cloud, viewers will learn the ins and outs of many of the services that make up the AWS cloud infrastructure. DevOps in the Cloud LiveLessons includes contributions by Brian Jakovich, who is a Continuous Delivery Engineer at Stelligent.

DevOps in the Cloud LiveLessons

Visit www.devopscloud.com to download the complete continuous delivery platform examples that are used in these LiveLessons.

Lesson 1:
Deploying a Working Software Application to the Cloud provides a high-level introduction to all parts of the Continuous Delivery system in the Cloud. In this lesson, you’ll be introduced to deploying a working software application into the cloud.

Lesson 2:
DevOps, Continuous Delivery, Continuous Deployment and the Cloud covers how to define motivations and differentiators around Continuous Delivery, DevOps, Continuous Deployment and the Cloud. The lesson also covers diagramming software delivery using spaghetti diagrams and value-stream maps.

Lesson 3:
Amazon Web Services covers the basics of the leading Infrastructure as a Service provider. You’ll learn how to use the AWS Management Console, launch and interact with Elastic Compute Cloud (EC2) instances, define security groups to control access to EC2 instances, set up an elastic load balancer to distribute load across EC2 instances, set up Auto Scaling, and monitor resource usage with CloudWatch.

Lesson 4:
Continuous Integration shows how to set up a Continuous Integration (CI) environment, which is the first step to Continuous Integration, Continuous Delivery and Continuous Deployment.

Lesson 5:
Infrastructure Automation covers how to fully script an infrastructure so that you can recreate any environment at any time utilizing AWS CloudFormation and the infrastructure automation tool, Puppet. You’ll also learn about the “Chaos Monkey” tool made popular by NetFlix – a tool that randomly and automatically terminates instances.

Lesson 6:
Building and Deploying Software teaches the basics of building and deploying a software application. You’ll learn how to create and run a scripted build, create and run a scripted deployment in Capistrano, manage dependent libraries using the Bundler and an Amazon S3-backed repository, and deploy the software to various target environments including production using the Jenkins CI server. You will also learn how anyone on the team can use Jenkins to perform self-service deployments on demand.

Lesson 7:
Configuration Management covers the best approaches to versioning everything in a way where you have a single source of truth and can look at the software system and everything it takes to create the software as a holistic unit. You’ll learn how to work from the canonical version, version configurations, set up a dynamic configuration management database that reduces the repetition, and develop collective ownership of all artifacts.

Lesson 8:
Database, covers how to entirely script a database, upgrade and downgrade a database, use a database sandbox to isolate changes from other developers and finally, to version all database changes so that they can run as part of a Continuous Delivery system.

Lesson 9:
Testing, covers the basics of writing and running various tests as part of a Continuous Integration process. You’ll learn how to write simple unit tests that will run fast tests at the code level, infrastructure tests, and deployment tests – sometimes called smoke tests. You will also learn how to get feedback on the test results from the CI system.

Lesson 10:
Delivery Pipeline demonstrates how to use the Build Pipeline plug-in in Jenkins to create a delivery pipeline for the commit, acceptance, load & performance and Production stages so that software can potentially be delivered to users with every change.


Continuous Delivery with Jenkins, CloudFormation and Puppet Online Course

This 3-hour online course is ideal for Developers or Sys Admins who want to know how to script a complete Continuous Delivery platform in the Cloud using Jenkins, AWS and Puppet. By the end of the course, you will have a working Continuous Integration system on AWS with all of the source code. The course is led by award-winning author of Continuous Integration and Stelligent CTO, Paul Duvall. The next course is Tuesday August 14, 2012 at 1PM EDT.

Attendees should be at least a junior-level Developer or Linux Sys Admin

In the course, you will:

  1. Use AWS CloudFormation to script the provisioning of AWS resources such as EC2, Route 53 and S3
  2. Script the entire Jenkins and target environment infrastructure using Puppet
  3. Create a fully-functioning Continuous Delivery platform using Jenkins and AWS
  4. Participate in discussions on Continuous Delivery patterns and best practices

You will also be able to email the trainer after the course with questions on setting up your CI platform in AWS.

Register now

There will be 15+ separate exercises that you will go through in creating a working Jenkins system in AWS. If you do not have an AWS account, you will need sign up by registering with Amazon Web Services as a part of the course pre-work (a credit card to pay AWS is required). The course material will be provided to students prior to the session. You will spend approximately $10 on AWS fees during the 3-hour training session.

You will be provided access to all of the source code and instructional material prior to, during and after the course If you have any questions, contact us at training@elasticoperations.com. A website with some of the example content for the course is located at http://onlinecd.s3-website-us-east-1.amazonaws.com/


Screencast on Scripted Infrastructure with AWS CloudFormation and Chef