Using rake to automate heroku tasks

One of the cool things that I’ve learned at ThoughtWorks is automating stuff. Automating daily tasks is pretty cool and important. For instance, all the teams that I’ve worked in the past 3 years create scripts to automate builds, deploys and database backups between other tasks.

At Motonow we try to do the same. Because we use Heroku infra-structure we have to remember a lot of the toolbelt’s commands in order to deploy the app, run database migrations and backup the database. Although the commands are pretty simple, we found ourselves running theses commands in a specific order and many times a day, which bother us a lot.

To avoid that, we decided to create rake tasks to automate some of the things we use frequently.

Deploying

Motonow deployment process is pretty straightforward. It’s basically composed of three steps at the time of the writing:

  1. Pushing code to Heroku;
  2. Running database migrations;
  3. Restarting the app;

Because we go to production many times every day, we would like to have all three steps automated. So we decided to write some rake tasks for that. The first step is creating an environment variable with the application name:

Rakefile

ENV["production_app"]   = "your_app_name"
ENV["staging_app"]      = "your_staging_app_name"

This is useful so you don’t have to duplicate your app name throughout your code. The next step is create the task itself.

lib/tasks/deploy.rake

namespace :deploy do

  desc 'Deploys a branch to staging. Use DEPLOY_BRANCH to specify which branch to deploy.'
  task :staging do
    Rake::Task["deploy:environment"].invoke("staging")
  end

  desc 'Deploys a branch to production. Use DEPLOY_BRANCH to specify which branch to deploy.'
  task :production do
    Rake::Task["deploy:environment"].invoke("production")
  end

  task :environment, :env do |t, args|
    deploy_branch(ENV["DEPLOY_BRANCH"], args.env)
    Rake::Task["heroku:migrate"].invoke(ENV["#{args.env}_app"])
    Rake::Task["heroku:restart"].invoke(ENV["#{args.env}_app"])
  end

  def deploy_branch(branch, environment)
    if branch
      sh "git push #{environment} #{branch}:master"
    else
      puts "Please, specify a branch to deploy."
      puts "Usage => deploy:staging DEPLOY_BRANCH=release_0.15"
      exit 1
    end
  end

end

lib/tasks/heroku.rake

namespace :heroku do

  task :migrate, :app_name do |t, args|
    run_command("rake db:migrate --trace", args.app_name)
  end

  task :restart, :app_name do |t, args|
    run_command("restart", args.app_name)
  end

  def run_command(cmd, app_name)
    Bundler.with_clean_env do
      sh build_command(cmd, app_name)
    end
  end

  def build_command(cmd, app_name)
    "heroku #{cmd} --app #{app_name}"
  end

end

A couple of things to mention:

  1. We decided to create a heroku.rake file that contains all Heroku specific commands.
  2. deploy.rake:21 works because our .git/config contains a remote named production/staging pointing to heroku repository.

Other than that, I believe the code is self-explanatory. Now to deploy the app, we run the following command:

rake deploy:production DEPLOY_BRANCH=branch_to_deploy

Copying database to staging

Another task we do a lot is copying production data to our staging app. This is useful to run some validation scenarios with real data and make sure that a new release don’t break the app. First, we need to understand how this process works in Heroku. There are two links that can help with that here and here. As you can notice, we need to run a couple of commands to achieve what we want, which means it’s a good candidate for automation.

In addition to copying, we would like to setup some data as well. For instance, delete the entries from a table called DEVICES and reset all user passwords. In short, the production refresh process includes:

  1. Moving data from one app to another in Heroku;
  2. Deleting data from DEVICES table;
  3. Reseting users passwords;
  4. Restarting the app.

See below the changes we came-up with to accomplish all that.

Rakefile

ENV["production_app"]   = "your_app_name"
ENV["staging_app"]      = "your_staging_app_name"
ENV["staging_database"] = "HEROKU_POSTGRESQL_PURPLE"

lib/tasks/db.rake

namespace :db do

  task :delete_devices => :environment do
    puts "Deleting driver devices"
    ActiveRecord::Base.connection.execute("DELETE FROM devices")
  end

  task :reset_passwords => :environment do
    puts "Reseting user passwords"
    ActiveRecord::Base.connection.execute("UPDATE users set encrypted_password = 'default_password'")
  end

  namespace :staging do

    desc 'Copies production data to staging database.'
    task :prod_refresh do
      Rake::Task["heroku:pgbackups:restore"].invoke(ENV["staging_app"], ENV["staging_database"])
      Rake::Task["heroku:rake"].invoke("db:staging:setup_data", ENV["staging_app"])
      Rake::Task["heroku:restart"].invoke(ENV["staging_app"])
    end

    task :setup_data => [:delete_devices, :reset_passwords]

  end

end

lib/tasks/heroku.rake

namespace :heroku do

  task :migrate, :app_name do |t, args|
    Rake::Task["heroku:rake"].invoke("db:migrate --trace", args.app_name)
  end

  task :restart, :app_name do |t, args|
    run_command("restart", args.app_name)
  end

  task :rake, :cmd, :app_name do |t, args|
    run_command("run rake #{args.cmd}", args.app_name)
  end

  def run_command(cmd, app_name)
    Bundler.with_clean_env do
      sh build_command(cmd, app_name)
    end
  end

  def run_command_with_output(cmd, app_name)
    Bundler.with_clean_env do
      `#{build_command(cmd, app_name)}`
    end.gsub("\n", "")
  end

  def build_command(cmd, app_name)
    "heroku #{cmd} --app #{app_name}"
  end

  namespace :pgbackups do
    task :restore, :app_name, :database_name do |t, args|
      production_url = run_command_with_output("pgbackups:url", ENV["production_app"])
      run_command("pgbackups:restore #{args.database_name} '#{production_url}' --confirm #{args.app_name}", args.app_name)
    end
  end

end

Here are the things to notice:

  1. New environment variable staging_database in the Rakefile;
  2. New rake file lib/tasks/db.rake that contains the code responsible for the production refresh process;
  3. New task :rake inside heroku.rb:11 which runs rake tasks for Heroku applications;
  4. New task pgbackups:restore in heroku.rb:32 which copies data from one app to another in Heroku;
  5. We refactored the task :migrate in heroku.rb:3 to use the new :rake task created;

Now, to copy data from production to staging environment, all we need to do is run the following command:

rake db:staging:production_refresh

Conclusion

As you can see, automating some of the tasks that you perform frequently can be very useful and productive. So if you find yourself running commands repeatedly throughout the day, try do create a script to do it for you. Hope the samples are useful to your team as they are for us.

 

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s