Dual Booting with Engines and Gems

Gems are a central part in a Rails application, they help us add new functionality to our apps so we don’t have to reinvent the wheel, but also allows us to extract code to better organize the codebase and to share logic between multiple apps. In many cases, we have custom made gems, and we need to ensure they will work properly with the two Rails versions we run when we use the Dual Boot technique opens a new window during upgrades. But… How do you dual boot the gems?

The Problem

When we think about dual booting an application, we usually target specific versions of a gem in our main Gemfile when we know which one is compatible with the different Rails versions we are using. For example, in one of our applications we have this:

# Gemfile

if next?
  gem "jekyll", "~> 4.0.1"
else
  gem "jekyll", "~> 3.7.4"
end

We know that we need a different version of the jekyll gem for each case.

The problem comes when the gem is not a public gem but an internal gem that is not yet compatible with the new Rails version.

Engines opens a new window are a more specific case of a gem, so we’ll talk about gems in this article and the same ideas apply.

Non-Dual Booting Approaches

If the gem lives in a separate repository, one approach we can take is to update the gem’s main branch to only work with the new Rails version and then use the if next? conditional to use different versions/commits/tags for each Rails version:

if next?
  gem "custom-gem", github: "my/repo", ref: "some-commit-sha"
else
  gem "custom-gem", github: "my/repo", ref: "some-older-commit-sha"
end

This is the simplest approach, but the caveat is that, during the upgrade process, we won’t be able to submit fixes or updates to the gem used by the current Rails version, since the main branch will only be compatible with the new version of Rails.

To prevent that, another slightly different approach is to create a separate branch in the gem’s repository, the main branch being compatible with the current Rails version and the new branch being compatible with the new Rails version and it gets merged at the end of the upgrade. Then, in the Gemfile, we can specify different branches for the source of the gem:

if next?
  gem "custom-gem", github: "my/repo", branch: "new-rails"
else
  gem "custom-gem", github: "my/repo", branch: "main"
end

This allows us to continue adding code to the gem that’s used by the app in production. But the caveat, in this case, is that we have to keep the branches in sync, so the new branch also includes the changes done in the main branch.

Dual Booting a Gem

If you noticed, those are the same two problems Dual Booting aims to solve when upgrading a Rails application, so we can borrow the dual booting idea and apply it to a gem: The gem adds compatibility with the next Rails version, without losing compatibility with the current one, allowing us to modify the gem at any point during the upgrade process ensuring that the latest version of the gem is compatible with both Rails versions.

Loosen the Rails Restrictions

The first limitation we typically see during our upgrades is the gemspec file of the gem adding a strict dependency on the current Rails version. For example, in a Rails 5.2 application we tend to see custom gems with something like this in its gemspec:

# custom_gem.gemspec

s.add_dependency "rails", "~> 5.2.4"

If we try to dual boot the main application with Rails 6.0, Bundler opens a new window won’t be able to resolve a matching version of custom_gem that is compatible, so we have to loosen this dependency. We don’t go too far, like with dual booting, we move up to the next minor version:

# custom_gem.gemspec

s.add_dependency "rails", ">= 5.2.4", "< 6.1.0"

We allow any patch version newer than Rails 5.2.4 and up to any patch version of Rails 6.0

Now Bundler will be able to resolve this because our gem reports being compatible with both Rails 5.2 and 6.0. But this is not enough, we have to actually make the gem compatible with Rails 5.2 and 6.0.

Note that we DID NOT add conditionals in the gemspec file! we will add that in the corresponding Gemfile.

Conditional Code

When dual booting, we ideally want to update the code to be compatible with the new Rails version in a way that the same also works with the current Rails version (we call them backwards-compatible changes). This is not always possible, so in many cases we have to add a conditional to execute different code for each Rails version. We can do the same in the gem:

if Rails.version ~= /^6.0/
  # some rails 6.0 specific code
else
  # some rails 5.2 specific code
end

Test Suite Inside the Engine

Testing With a Dummy App

If the gem includes standalone tests (tests that run independent from the main Rails application), we typically test them using a dummy Rails application (this is really common for Engines). When that’s the case, we can dual boot the dummy application the same way we dual boot the main application, and then run the tests once for each Rails version. That way we can test that our code changes are safe in both Rails versions.

Tests Without a Dummy App

In some cases, the dependency is not with the Rails gem but with some railtie (like active_support for example). In those cases the gem probably won’t have a dummy Rails application to run the tests since it doesn’t need a complete app, but it can still have tests. In these cases, we also have a Gemfile and a Gemfile.lock, and we can apply the dual boot idea here. There’s one detail to keep in mind… in this cases, the Gemfile may look like this:

# Gemfile
source "https://rubygems.org"

gemspec

We can still dual boot this, we can create the Gemfile.next symlink and add conditionals:

# Gemfile
source "https://rubygems.org"

def next?
  File.basename(__FILE__) == "Gemfile.next"
end

gemspec

if next?
  gem "active_support", "~> 6.0.5"
else
  gem "active_support", "~> 5.2.4"
end

The strict version specified in the conditionals will tell Bundler which version of the game to use for each Gemfile.lock file. This file is not used by the main application when using the gem, this is only used for the gem’s standalone tests (or when the gem is meant to also be run as a standalone application).

Note that the versions used there must be within the range specified in the gemspec.

Then we can generate the new Gemfile.lock.next (don’t do it from scratch! opens a new window ) and run the tests with different sets of gems:

bundle exec rspec

# and

BUNDLE_GEMFILE=Gemfile.next bundle exec rspec

Updated Gemfile.lock

Since the Gemfile changed, we also need running bundle install to keep the Gemfile.lock in sync, because the gem’s dependencies restrictions changed. We don’t want to change any other gem except the one we are updating, so the changes in the Gemfile should be minimal and only reflect the new loosen restrictions and nothing more. For example, this is the expected change:

    custom_gem (0.1.0)
-      rails (= 5.2.4.4)
+      rails (>= 5.2.4, <= 6.1.0)

There should be no update for the Rails version gem in the Gemfile.lock file, gems versions should only be updated in the Gemfile.next.lock opens a new window . We want to make sure the application setup is not modified by our changes during the upgrade for the current Rails version.

Testing the Engine in the Main App Test Suite

In some cases, the gems don’t have standalone tests and the functionality is tested within the main application test suite. When that happens, we use those tests to verify the compatibility.

Custom Gems Inside the Main App Repo

When engines and gems live inside the main application repository, we can only apply the dual boot technique, since we can’t target a gem in the same source using a different commit hash or branch.

Bonus Tip: Support More Rails Versions

In some cases, a gem needs to be updated to support more than one Rails version. We can still apply the same technique, we use the next naming convention because it fits a dual boot scenario, but we can always use the same idea and use a different name for the Gemfile.next for each Rails version, and we can loosen the dependency even more:

We can have the main Gemfile file, then symlink a Gemfile-5-2, Gemfile-6-0, Gemfile-6-1 (and so on), and we can use the BUNDLE_GEMFILE env variable to pick which tests to run and to add conditionals in the Gemfile.

Conclusion

Rails upgrades for applications with custom gems have been more common for us over the last few months, and it is important for us to handle upgrades in a way that is less disruptive to our clients’ workflow. Being able to dual boot the custom gems so our clients can continue working on them during the upgrade process and, at the same time, not having to keep an extra upgrade branch in sync is a win for all. It also gives the client the confidence that the changes we are doing are always compatible, and, if they need to change something on that gem during the upgrade, they can already test their changes against the next Rails version without waiting for the upgrade, helping them adapt their code for the future.