I’m a huge proponent of automating repetitive processes. One process that’s usually a good one to automate in a software project is how to deploy the software.
As an example of how to do this, I’m going to use my own website.
The Manual Process
The first step to figuring out how to automate a process is to figure out what the manual version of the process would be.
My website’s source code lives in a Git repository. The first step in the deployment process is to make sure that I haven’t made any accidental changes. Perhaps you did a find and replace that made changes you didn’t expect. Maybe you started rewriting a paragraph and didn’t finish. For me, if a change hasn’t been committed to Git yet it’s still a work in progress, and I shouldn’t be deploying it yet. If there are uncommitted changes, they need to be dealt with before the deploy can proceed.
The next step is to make a production build of my website. The
production configuration turns on some features, like Google
Analytics, that I don’t want while I’m testing the website on my
computer. This involves opening a terminal and typing
JEKYLL_ENV=production jekyll build.
The final step is to open up an FTP client and copy the files from my build to the web server. This is a fairly simple process as far as website deployments go. My website is just a static collection of files after it’s been built, so there’s no extra steps to actually start it up. If the files are in the right folder, it works.
What Could Go Wrong?
Apart from just being a pain to go through, doing a deployment manually is a bit dangerous. What if I get distracted and miss a step? What if I set the Jekyll environment to “poduction” instead of “production” and accidentally deploy a site without analytics enabled? What if I miss a file when selecting what to upload over FTP? Writing a script to do the deployment for me solves these problems, as well as saving me time.
Just Automate It
In a team setting, you’d usually have a server set up running Jenkins, TeamCity, or some other specialized tool for running these jobs. On other projects, I do. In this case however, I’m just one person working on the website, so it’s enough for me to have a shell script that lives in the project. It does all of the steps above, but will do them the same way, reliably, every time.
In terms of security, the script currently needs to be run in an interactive shell so that it can prompt me for my FTP password.
#!/bin/bash FTP_USERNAME='your-username' FTP_URL='sftp://your-ftp' TARGETFOLDER='public_html' SOURCEFOLDER='_site/' cd `dirname $0` echo "Working directory: `pwd`" if [[ `git status --porcelain` ]]; then echo "There are uncommitted changes." echo "Please commit any changes before deploying." exit 1 fi echo "Running production build" JEKYLL_ENV=production jekyll build || exit 1 echo "Deploying $SOURCEFOLDER to $TARGETFOLDER on $FTP_URL" lftp -f " open -u $FTP_USERNAME -e ls $FTP_URL || exit 1 mirror --reverse --delete --depth-first --overwrite --verbose $SOURCEFOLDER $TARGETFOLDER || exit 1 bye " || exit 1 echo "Deploy completed successfully"
Some important points that I’d like to call out.
- If something goes wrong, like the Jekyll build fails, this script
aborts. It won’t upload an empty folder. This is done by the
|| exit 1after each command.
- We’re running an FTP script in our script. It also needs to exit early if, for example, your password is incorrect.
- Status updates are printed to the console. This is important when you’re watching the script and not sure if it’s doing anything. Logging information in this way is also important for if something goes wrong, you can see where things failed and what was going on.
- We start by making sure we actually have the right directory. Don’t assume that the user is going to call it from the right place.
--depth-firstflag in my FTP command has the effect of making sure that all of my posts are uploaded before my homepage and RSS feed are updated. This is important. Nobody wants the ‘latest news’ at the top of their homepage to point to something that doesn’t exist yet.
My website is an anomaly in terms of projects I’ve worked on, partly because it’s so small and partly because it’s just static files being served up.
I can avoid perceived downtime by just being careful about the order that files are being uploaded. Your web application may need to worry about database connections or somebody in the middle of a wizard. An advantage of automating your deployments is that the processes can get more complicated to add features like Blue Green Deployments without actually making it more difficult for an individual on your team to run a deployment.
If you have a QA server, you can also set up a script that watches your Git repository, and automatically deploys any changes to QA as soon as they are pushed to your master branch. That, however, is a topic for a different post.