Deploy complex apps with Atomic SFTP deployments

Today we’re releasing something we’ve been looking forward to for years. We’re proud to announce a new way to deploy your web application (and not only web) that will help to make sure that your users do not experience even the slightest inconvenience. And if things do go wrong, they will not be affected. A noble goal, I hope you’ll agree.

Here’s what we’re trying to solve

First, let me explain what makes an application complex and why its deployment needs are special:

• They generally have a lot of moving parts, like additional services or dependencies.
• A lot of files need to be updated during the deployment.
• A lot of users actively use the app at all times, even while the deployment is happening.
• Special procedures need to happen during updates.
• You need to be able to roll back faulty changes quickly in case something goes wrong.
• A failure during a deployment should not impact your users.

All of these points illustrate that simply uploading files through your regular SFTP deployment process or running git pull on a server while the app is running is not a very good idea. What happens if the upload or a pull fails in the middle?

Our solution — atomic deployments

When you deploy a mission critical application you can’t just hope that everything is going to be all right when you click the Deploy button. You want to know you did everything possible to make sure it will be successful. For critical applications like this, we present a new type of deployments called Atomic SFTP.

Why atomic? Atomicity is a concept in database systems that describes a type of transaction that either occurs in its entirety or is discarded completely without a trace, not affecting anything. That describes the ideal deployment we want to have — it’s either done completely or discarded without affecting the running version of code at all.

The way we reach that goal is by following a specific set of steps during your deployment:

1. When you deploy we update the copy of your code in a cache directory on your server.
2. When the code is updated, we hardlink that code to a separate release directory.
3. In the release directory we execute all of the user-defined commands required to get code into shape — like compiling assets or fetching dependencies through bundler or composer.
4. Once the release is ready, the current symlink is updated to point to a new release directory.
5. User-defined commands are then executed to restart your application server or related services.

If even one of those steps fail, your current running code is unaffected, as we delay updating the current symlink until the very end of the deployment. Old releases are preserved in case you need to rollback in emergency, but are cleaned up for you when there are too many of them (this value is configurable).

Getting up and running

The first thing that you need to do is to get your free DeployBot account if you don’t have one yet. After connecting your repository and creating your first environment, you can now go on to creating your Atomic SFTP server:

Set up commands to be executed during deployment

In addition to just getting atomic deployments, you can also execute commands (with a range of special variables available) at certain points in the deployment. For example, making sure your assets are compiled and your log folder is linked properly:

Deploy config files

When you deploy, you often need to store a configuration file on your server or some other file that has to be present across your environment — like an SSL certificate. For that you can use DeployBot’s feature called Configuration files, which allows you to store and edit a set of files for environment and then configure each server to grab these files and deploy them to your servers:

Please note, that if you decide to deploy with Atomic SFTP deployments you will need to update your web server to look for the app in your new web root — /data/myapp/current in this example. With that your configuration process should be complete.

Need to roll back fast?

Now, in case when something bad happened, rollback is just one click away:

Need to deploy to many servers at once?

With environments you can deploy simultaneously to tens of servers and more. You also get great tools that allow you to quickly clone a server or put deployments to a certain server on pause temporarily.

You also get many other things

Notify your team using email, Slack, Hipchat, Campfire and integrate with New Relic to correlate your deployments with performance changes. You can also mix different kinds of deployments in the same environment and trigger them as part of the same process — for example you can deploy your assets to S3 while you’re deploying your app.

---

I would like to finish by saying that we really encourage you to give this way of deployments a try. It will make deployments easier and offer a more relaxed experience. If you already using a tool like Capistrano to manage your deployment consider how much easier it can be with DeployBot, especially given that now people without SSH access to the servers will be able to deploy.

An interesting fact about this release is that it all started with a blog post back in April that was never published. In that blog post I tried to describe how you can have atomic deployments working in DeployBot and then it turned out that it was not as straightforward as we wanted. We went back to the drawing board and added and changed many things, so that atomic deployments could finally be possible.

We hope this great new way to deploy will make your deployment what it has to be — a celebration of achievement, not a fear of terrible failure waiting to happen. If you have any other ideas about what can make your deployments more pleasant and less troublesome, as usual — we’re all ears.