How to dump the database whenever you deploy with Capistrano

We often want to do a database dump before deployment of new releases. This backs you up in case migrations touch your data in a unfavorable way or a new feature is buggy.

I already introduced you to our dumple script in order to dump from within rails projects. We often use it for this purpose. To do so, you need dumple on the remote machine. Additionally add two tasks to the database namespace and two callbacks in config/deploy.rb:

namespace :db do

  ...

  desc "Do a dump of the DB on the remote machine using dumple"
  task :dump do
    rails_env = fetch(:rails_env, 'production')
    run "cd #{current_path}; dumple --fail-gently #{rails_env}"
  end

  desc "Show usage of ~/dumps/ on remote host"
  task :show_dump_usage do
    run "dumple -i "
  end

end

...

before "deploy", "db:dump"
after "deploy", "db:show_dump_usage"

Next time you cap deploy you will see some output before and after the deployment. The first (db:dump) indicates whether the dump was successful and prints its size. The second one (db:show_dump_usage) shows the overall consumption of the dumps-directory.

If you still wonder about the --fail-gently parameter: database.yml does not exists when you run your first deployment of a project. If you raise within a Ruby script the exit code is 1 and Capistrano stops. The gently parameter makes dumple use exit, which results in a exit code of 0 and Capistrano continues!

Avatar

Tue, 17 Aug 2010 11:40:00 GMT

by thomas

Tags:

Aegis now supports multiple roles per user

Even though we sort of said we were never going to do it, Aegis now supports multiple roles per user. You can give a user multiple roles like this:

user.role_names = ['author', 'editor']

Permission is granted when at least one role is allowed access, even if other roles are not.

Avatar

Sun, 25 Jul 2010 10:49:00 GMT

by henning

Tags:

Mail Magnet makes your e-mails go where you want them

Remember that nifty way to override e-mail recipients we showed you a while ago? It evolved into a gem: Mail Magnet makes changing e-mail recipients for your staging environments even easier.

Just go with a sudo gem install mail_magnet and add something like this to your staging environment file:

config.gem 'mail_magnet'
config.after_initialize do
  ActionMailer::Base.override_recipients = 'overridden@example.com'
end

More information can be found at the gem’s github page.

  • Morgan Currie said about 1 month later:

    Hey guys, Great job on this gem! I was setting up our app to use mail_magnet in both development and staging environments today, but as we send all HTML e-mails, the output was getting smushed together. I forked mail_magnet and updated it to differentiate between text/plain and text/html content_types, with 3 new tests to verify the behaviour. Check it out here: http://github.com/morgancurrie/mail_magnet

  • Arne said about 1 month later:

    Everyone else wo is sending HTML e-mails can now get version 0.2.0 of Mail Magnet which includes Morgan’s changes.

Avatar

Fri, 23 Jul 2010 15:02:00 GMT

by arne

Tags:

How to warn of pending migrations when deploying with Capistrano

We use Capistrano for automatic deployment. A common problem is that you don't remember whether your current release includes migrations and you therefore have to run cap deploy:migrate to update your remote database after deployment. A small piece of code warns us if there are pending migrations:

namespace :db do

  desc "Warns if there are pending migrations"
  task :warn_if_pending_migrations => :environment do
    if defined? ActiveRecord
      pending_migrations = ActiveRecord::Migrator.new(:up, 'db/migrate').pending_migrations

      puts ""
      puts " * * * * * * * * * * * * * * * * * * * * * * * * * * * *"
      puts " *                                                     *"
      if pending_migrations.any?
        puts " * You have #{pending_migrations.size} pending migrations:"
        pending_migrations.each do |pending_migration|
          puts ' * %4d %s' % [pending_migration.version, pending_migration.name]
        end
        puts " *"
        puts " * Run cap deploy:migrate!"
      else
        puts " *               No pending migrations.                *"
      end
      puts " *                                                     *"
      puts " * * * * * * * * * * * * * * * * * * * * * * * * * * * *"
      puts ""
      
    end
  end

end

All you need to do is put this snippet into lib/tasks/pending_migrations.rb and add this line end the end of config/deploy.rb in your project:

after "deploy", "db:warn_if_pending_migrations"
  • Thomas said 3 months later:

    Thanks Nick, seems to be a good way if nothing but the migration changed. If there is more stuff that diffs it seems to be a bit confusing to me.

    Cheers

  • Ramon said about 1 year later:

    I get this error: the task `db:warn_if_pending_migrations’ does not exist

    If I remember correctly, cap tasks run in the machine that’s deploying. It seems you’re trying to run a rake task after deploy – which you may have to do something like http://stackoverflow.com/questions/312214/how-do-i-run-a-rake-task-from-capistrano

  • thomas said about 1 year later:

    Ramon, you’re right. Cap tasks run on the local machine you’re deploying from. The error you mention means that Capistrano is unable to find the task. Did you put the code snippet into the correct namespace (db)?

    Use

    # cap -T

    to get a list of all Capistrano tasks. It should look like that:

    ...
    cap db                 # Create database yaml in shared path
    cap db:dump            # Do a dump of the DB on the remote machine...
    cap db:show_dump_usage # Show usage of ~/dumps/ on remote host
    cap db:symlink         # Make symlink for database yaml
    cap db:warn_if_pending_migrations # Warn about pending migrations
    
    ...
    

    Cheers, Thomas

Avatar

Mon, 16 Aug 2010 17:49:00 GMT

by thomas

Tags:

Solving the N + 1 query problem with Query Diet

As developers we use O/R mappers to pretend our data is stored as objects instead of flat tables. Unfortunately that abstraction breaks down frequently. Or as Joel would say:

All non-trivial abstractions, to some degree, are leaky.

One particular leak of O/R mapping is the N + 1 query problem, where the most innocent code can flood your database with hundreds or thousands of queries.

N + 1 queries are easy to overlook during development because we are used to mediocre response time on our development machines with disabled caches and class reloading. Relief is promised by debug bars like Rack::Bug and rails-footnotes which include the number of triggered database queries in the HTML of every page coming out of your application. Unfortunately they also do a million other things we don't care about. We wanted something that would alert us of excessive queries, but stays out of our way otherwise.

Enter Query Diet.

Query Diet inserts a tiny, translucent box into the upper right corner of your screen, informing you about the number of queries triggered by the last request and the time spent waiting for the database.

This is Query Diet being happy about 3 requests taking 66ms:

Happy Query Diet

This is Query Diet being angry about 103 requests taking 164ms:

Angry Query Diet

We're also quiet satisfied with Query Diet's installation procedure, which goes like this:

  1. config.gem 'query_diet'

That's it. No second step required.

Check out Query Diet on github.

Avatar

Fri, 25 Jun 2010 16:23:00 GMT

by henning

Tags:

Callbacks for the Machinist

We like Machinist a lot. Unfortunately there are record construction rules cannot be expressed with a Machinist blueprint, for instance:

  • Creating a record with a has_many association already filled in.
  • Creating a record with interdependencies between associated objects, such as having to belong to the same container.
  • Needing to process a delayed_job queue after record construction.

We just released machinist_callbacks to fix that. This gem provides before_make and after_make callbacks, allowing you to write a blueprint like this:

Director.blueprint do
  name
  after_make { movies << Movie.make }
end

Check out machinist_callbacks on Github.

Avatar

Sun, 20 Jun 2010 12:17:00 GMT

by henning

Tags:

A gentler Cucumber Factory

Creating database records for a Cucumber scenario is painful. It involves writing a trillion steps for every model class and attribute in your app. That's why we released Cucumber Factory which does all the heavy lifting for you.

However, a common complaint with generic factory steps was that they grab too much of Cucumbers global pattern namespace. You could never again write a step definition that starts with "Given there is a..." because Cucumber would complain about ambiguous step patterns.

With the new Cucumber Factory 1.5.0 this is no longer something you need to worry about. Cucumber Factory now stays silently in the background, always preferrring your handwoven definitions. Only if none of your own step definitions match, Cucumber Factory's automagic implementation kicks in.

This also means that you can now add Cucumber Factory to your existing project without changing behavior.

Check out the new version on Github.

  • Dennis said about 4 hours later:

    Thank you, I’ll try it out!

  • Ed Jones said 11 months later:

    Wondering about this topic a year later. Pickle seems to have many recent commits, Cucumber Factory not so many. And little publicity of CF on the web.

    CF still an equal or better contender for someone just choosing a solution?

  • Henning said 11 months later:

    @Ed: Cucumber Factory only sees an occasional commit because we are quite happy with it as it is.

    Which gem you choose comes down to your personal preference – Pickle has a few more features, Cucumber Factory has more natural sounding steps. I recommend you try both and see which one suits your style.

Avatar

Sun, 20 Jun 2010 12:01:00 GMT

by henning

Tags:

Aegis 2 brings resources to your permissions

Yesterday we released the next major revision of our popular authorization solution Aegis.

Since the inception of Aegis in 2007, the way we write our Rails controllers has changed dramatically. We are now using RESTful routes and slim down controllers with resource_controller and modularity. We're getting more bang for each line of code. It was time for Aegis to catch up.

Aegis 2 lets you describe your permissions using resources, similiar to your routes. Your permission resources can match those in your routes, but don’t have to. Here is how your permission definitions can look like in Aegis 2:

class Permissions < Aegis::Permissions

  role :user
  role :admin

  resources :projects do
    allow :everyone
  end

  resources :users do
    allow :admin
  end

end

You can protect all actions in a controller through an Aegis resource with a single line:

class ProjectsController < ApplicationController
  permissions :projects
end

There are many more new features, including defining permissions for reading or writing actions and a way to check permissions when no user is signed in.

We also started an awesome documentation wiki for Aegis where you can find detailed information on many basic and advanced topics, including:

There is also a migration guide for people who want to bring their Aegis 1 setup up to speed.

Avatar

Mon, 03 May 2010 07:12:00 GMT

by henning

Tags:

How to use multiple versions of Cucumber in parallel

When you have multiple Rails projects running different versions of Cucumber, you have seen this error:

0.6.4 is not a class/module (TypeError)

Or maybe you had fun with this one:

can't activate cucumber (= 0.4.0, runtime) for [], already activated cucumber-0.6.4 for [] (Gem::LoadError)

It requires some tweaks to get multiple versions of Cucumber to play nice with each other. Also since there are three distinct ways to invoke Cucumber in a Rails project, you will need a slightly different hack for each of them.

The first way to invoke Cucumber is by calling rake cucumber. To make rake aware of the correct version, add the following line to the top of lib/tasks/cucumber.rake, replacing 0.4.0 with the desired Cucumber release:

gem 'cucumber', '=0.4.0' if Gem.available?('cucumber')

Now that rake does our bidding, we can move on. When running invididual features RubyMine uses script/cucumber rather than rake, so we will need to fix that as well. Add the following to the top of script/cucumber, right below the shebang:

require 'rubygems'
gem 'cucumber', '=0.4.0' if Gem.available?('cucumber')

Finally, some people (me!) like to run features using the cucumber executable, which uses the latest Cucumber by default. These people can either switch to rake or script/cucumber, which we already fixed. Or, if you absolutely, positively need to continue using the cucumber executable, you can request a specific Cucumber version via command line parameter:

cucumber _0.4.0_ features/articles.feature

These are tweaks that have worked for us. There are other options that involve freezing Cucumber or using the Ruby Version Manager, but we like the leightweightness of our approach.

Did you find this article helpful? Let us know in the comments.

  • Dominik said about 1 month later:

    Wow thanks a lot! This was exactly what I needed.

Avatar

Wed, 21 Apr 2010 14:13:00 GMT

by henning

Tags:

Never write a Cucumber step definition again with Cucumber Factory

I love Cucumber, but I hate writing step definitions. They are ugly, awkward to write and very, very boring: Most step defintions merely create objects for a scenario to chew on.

Eventually we created Cucumber Factory so we would never have to write that sort of step definition again. Cucumber Factory takes this scenario line:

Given there is a movie with the title "Sunshine" and the year "2007"

And transforms it into this:

Movie.create(:title => "Sunshine", :year => "2007")

Cucumber Factory comes with many little helpers to deal with complex cases, while always looking awesome. It also plays nice with your Machinist or Factory Girl.

Check out Cucumber Factory on GitHub.

Avatar

Sat, 17 Apr 2010 19:25:00 GMT

by henning

Tags: