No fear Rails schema changes

Over on the PlanetScale blog I just wrote up how we handle schema changes in our own Rails app.

It's the best workflow I've ever used for Rails migrations. It's both inspired by and better than the custom tooling I had access to when working for GitHub.

Click the big button to check it out.

Why this is important to me

I remember my first Rails job and having to memorize which schema changes were safe to make in production without downtime. Which could lock a table. Which would break the Rails app.

It was stressful! Often it was easier to just take the app offline, run the migrations and bring it back up than deal with it.

I remember doing crazy things in my apps just to avoid changing the database. Even adding json columns so I could stuff whatever I wanted in them.

The "online" schema change

Then I went to work for GitHub and learned a better way. GitHub used a tool called gh-ost. This is known an a "online schema change" tool. Online means your app stays online while it runs (very to the point!).

It works because instead of running the schema change directly on the table. It creates a 2nd version of the table, runs the migration and then swaps them in place.

This solves the fear!! No longer did I need to worry about what my migration would do to production traffic.

Reverting changes

Since then, there has been a lot of innovation in the area of schema changes. Databases have improved and now support more "Instant" operations (meaning no table locking). There is still a lot of value though in the "ghost table" trick used by gh-ost.

The Vitess team has innovated on this pattern and added the ability to quickly "revert" back a schema change if it causes a problem. This is sounds simple. But it's absolutely mind bendingly cool. Because it does this without losing any data. They aren't simply running the reverse migration. They are instead keeping that old table around and running all writes/updates against it. You have both the old and new version of the table still in prod and still getting updates!

Then the revert scenario is simply swapping back to the old version. It's incredible.