Automating Deployment with Git

This tutorial provides step-by-step instructions to:

  • Install Git on a Linux server (commands are provided for Ubuntu and CentOS).
  • Pull source code from a remote Git repository onto the server.
  • Create a shell script to automate Git pulls.
  • Setup a cron task to pull in the latest code periodically, enabling continuous deployment.

How to install Git

On Ubuntu, use the apt package manager.

sudo apt-get update
sudo apt-get install git

On CentOS, use the yum package manager.

sudo yum update
sudo yum install git

How to configure Git

The Git client is configured on a user-by-user basis. I recommend that you create a Linux user that will be dedicated to the job of continuously integrating source code from a remote Git repository. This user should be given read, write and execute permissions to the /var/www directory. You can enable those permissions by adding the user to the www user group.

SSH into your Linux box as the user that will be responsible for the continuous integration process.

The first step is to configure the git command line tool for the current user. However, since the user will be mostly pulling from a remote Git repository, not pushing to that repository from the local machine, this step is not strictly necessary. Nevertheless, I like to configure things properly in the event that I do need to make commits to the upstream Git repository, for example in the event of hotfixes being applied directly on the server. So, type the following commands to provide the local Git user's name and email address. This information will be automatically attached to every local Git commit that is made by the current Linux user. The information does not need to match anything else, all it does is help to identify the author of commits made in local Git repositories. I tend to use a configuration similar to the following.

git config --global user.name "Hostmaster"
git config --global user.email "[email protected]"

Use the following command to check the configuration.

git config --list

To make further changes, you can directly edit the .gitconfig file for the current Linux user.

nano ~/.gitconfig

Initialize a local Git repository

Navigate to your web application's document root (as configured in the web server).

cd /var/www/www.example.com

Ideally, the host directory should be empty from the outset. Any files and directories that are required by the local host environment or that are dynamically generated on the server – things like logs, caches, uploads – must be excluded from version control. Place a .gitignore file in the root of the application's Git directory (do not install this file directly on the server). Below is a template. Be sure to include rules that cover everything that may exist within the application's root directory but which is not part of the application's source. If you do not do this, the git pull operations will fail whenever you have local changes that would be overwritten by the merge.

.cvs
.DS_Store
.svn

*.bak
*.log
*.swp

Thumbs.db

/node_modules
/private/logs
/private/sessions
/tmp
/vendors

Initialize the local Git repository:

git init

The output should read:

Initialized empty Git repository in /var/www/www.example.com/.git/

Next, you need to link the local repository to the remote repository, from where the application's files will be pulled:

git remote add origin <remote_repository>

Replace <remote_repository> with the URL to the upstream repository's .git file. The URL is provided by GitHub, GitLab, Bitbucket, or whatever distributed version control system your are using to host your upstream repository. Provide the SSH version of the URL rather than the HTTPS version. Using SSH to connect to the repo allows you to use SSH keys to act as proxies for your GitHub/GitLab/Bitbucket username and password, so you don't need to enter these whenever you connect to the remote repository. This makes automation possible. Example:

git remote add origin ssh:[email protected]/username/repo.git

Check carefully the syntax of the SSH address. This is invalid:

ssh:[email protected]:username/repo.git

This is correct:

ssh:[email protected]/username/repo.git

Verify the remote URL:

git remote -v

The output should be similar to this:

origin  ssh:[email protected]/username/repo.git (fetch)
origin  ssh:[email protected]/username/repo.git (push)

Generate SSH keys

The next step is to generate SSH keys that will act as proxies for your GitHub/GitLab/Bitbucket username and password.

Type:

ls -la ~/.ssh

If you see an existing public and private key pair listed (id_rsa.pub and id_rsa), then your Linux user account already has SSH keys configured for it to act as an SSH client. Otherwise, to generate new keys:

cd ~/.ssh
ssh-keygen -t rsa -b 4096

When you're prompted to "Enter a file in which to save the key," press Enter to accept the default file location.

At the prompt, press Enter to skip the input of a key passphrase. This is important. If you want to automate pulls from a remote Git repository, the automation scripts cannot be prompted to enter passwords.

Now, you need to copy the user's public key to your central Git host. To output the contents of your public key:

cat ~/.ssh/id_rsa.pub

Copy the the public key into the appropriate setting in your Github or Bitbucket account. In GitHub, you add the public key as a "Deploy key" in the settings for the appropriate repository. In Bitbucket, you add the public key as an "Access key" in the relevant repository's settings.

Now try pulling in the latest changes from the repote repository:

cd /var/www/www.example.com
git pull origin master

The response should be something like this:

The authenticity of host 'bitbucket.org (104.192.143.2)' can't be established.
RSA key fingerprint is SHA256:zzX......
Are you sure you want to continue connecting (yes/no)?

Type "yes". The host will be whitelisted, so you will never see this warning again when you issue any git commands over SSH.

Warning: Permanently added 'bitbucket.org,104.192.143.2' (RSA) to the list of known hosts.

After this message you should see the normal git output, which should confirm that everything is working. If your web application is configured correctly, it should be running on the server, too.

Create a shell script to automate

It's now a trivial matter to create a shell script to automate pulls from your Git repo. We'll place this shell script in a directory called "tasks" in the user's home directory.

cd ~/
mkdir tasks
cd tasks
nano deploy-example-app.sh

Add the following content.

#!/bin/bash
cd /var/www/www.example.com
git pull origin master

Allow execution of the script:

chmod +x deploy-example-app.sh

Now, to run this script manually, from the command line:

sh ~/tasks/deploy-example-app.sh

Check it is working by commiting a change the remote repository's master branch, then running the shell script. Your changes should be reflected immediately in the running web application.

Set up a cron task

To edit the the cron tasks delegated to the current Linux user, type:

crontab -e

Add the following contents to the cron config file, replacing [email protected] and <username> as appropriate.

MAILTO="[email protected]"

# Minute  Hour     DayOfMonth  Month  DayOfWeek  Command
# 0-59    0-23     1-31        1-12   0-6

*/5       08-17    *           *      1-5        /home/<username>/tasks/deploy-example-app.sh >/dev/null 2>&1

The last line runs the deploy-example-app.sh script every five minutes between the hours of 8am and 5pm, Mondays through to Fridays. The times are relative to the server clock's timezone. Assuming that the server clock's timezone is UTC, the script will be run between the hours of 9am and 6pm British Summer Time.

By default, any output from of any of the cron scripts will be emailed to [email protected]. However, this is disabled for the deploy-example-app.sh script, because appending >/dev/null 2>&1 to the end of the command path redirects the output to a null value.

When you save and exit the cron task configuration file, the response should be:

crontab: installing new crontab

Check it's working by making some commits to your master branch and then waiting for the changes to magically trickle through to your web application.