Part 3/3 - Deploying with git - Adding builds
We've setup a basic git push
workflow, and then taken it further by extracting out some of the common / important files and folders, and used the symlink strategy to share things between deploys.
Now we're going to introduce the concept of builds.
Just like in the rsync
improved video, we will create the concept of a builds
directory which we can then store several of our most recent builds, and use a symlink to set the currently live build as desired.
When working with git builds, personally I would prefer to use a commit hash as a build reference. This is somewhat bizarrely, much more difficult than you might expect.
If you are aware of an easy way to get access to the current commit hash when running the post-receive
hook then please do share.
As it stands, we are going to go with a 'timestamp' from the time when we run git push
.
Getting Setup
Much like previously, I'm going to assume that you are working in a controlled demonstration environment, and that the simplest way to 'reset' is to delete everything and start from fresh.
If you are following this based on some real world setup then adjust accordingly, and for the love of Mike, take and test your backups before proceeding.
# from the server
ls -la
total 16
drwxr-xr-x 4 root www-data 4096 Nov 18 13:04 .
drwxr-xr-x 5 root root 4096 Nov 18 11:07 ..
drwxrwxr-x 9 root root 4096 Nov 18 13:17 code
drwxrwxr-x 4 www-data www-data 4096 Nov 18 12:34 shared
rm -rf code/
mkdir builds
ls -la
total 16
drwxr-xr-x 4 root www-data 4096 Nov 18 13:45 .
drwxr-xr-x 5 root root 4096 Nov 18 11:07 ..
drwxrwxr-x 2 root root 4096 Nov 18 13:45 builds
drwxrwxr-x 4 www-data www-data 4096 Nov 18 12:34 shared
We already have most of the pieces in place to solve this puzzle.
The only additional steps we need to take are in 'timestamping' the new build, and updating the symlink for current
.
Oh, and updating the webserver configuration appropriately :)
Let's start with the easy bit.
Webserver Configuration
In order to see our changes we need to make sure the web server config is pointed at our revised directory structure. We will be using the symlink of current
to point to our desired build:
For Apache:
<VirtualHost *:80>
ServerName crvfakeexample.com
ServerAlias www.crvfakeexample.com
DocumentRoot /var/www/crvfakeexample.com/current/web
<Directory /var/www/crvfakeexample.com/current/web>
AllowOverride None
Order Allow,Deny
Allow from All
<IfModule mod_rewrite.c>
Options -MultiViews
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ app.php [QSA,L]
</IfModule>
</Directory>
# uncomment the following lines if you install assets as symlinks
# or run into problems when compiling LESS/Sass/CoffeeScript assets
# <Directory /var/www/crvfakeexample.com/current>
# Options FollowSymlinks
# </Directory>
# optionally disable the RewriteEngine for the asset directories
# which will allow apache to simply reply with a 404 when files are
# not found instead of passing the request into the full symfony stack
<Directory /var/www/crvfakeexample.com/current/web/bundles>
<IfModule mod_rewrite.c>
RewriteEngine Off
</IfModule>
</Directory>
ErrorLog /var/log/apache2/crvfakeexample_com_error.log
CustomLog /var/log/apache2/crvfakeexample_com_access.log combined
</VirtualHost>
Don't forget to reload Apache:
sudo service apache reload
And for nginx:
server {
listen 80 default;
server_name crvfakeexample.com www.crvfakeexample.com;
root /var/www/crvfakeexample.com/current/web;
location / {
# try to serve file directly, fallback to app.php
try_files $uri /app.php$is_args$args;
}
# DEV
# This rule should only be placed on your development environment
# In production, don't include this and don't deploy app_dev.php or config.php
location ~ ^/(app_.*|config)\.php(/|$) {
fastcgi_pass unix:/run/php/php7.0-fpm.sock;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $realpath_root;
}
# PROD
location ~ ^/app\.php(/|$) {
fastcgi_pass unix:/run/php/php7.0-fpm.sock;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $realpath_root;
internal;
}
# return 404 for all other php files not matching the front controller
# this prevents access to other php files you don't want to be accessible.
location ~ \.php$ {
return 404;
}
error_log /var/log/nginx/crvfakeexample.com_error.log;
access_log /var/log/nginx/crvfakeexample.com_access.log;
}
Don't forget to reload nginx:
sudo service nginx configtest
sudo service nginx reload
Basically a find / replace exercise.
Builds
Things get a little more complicated at this point.
Here's the new script in full:
#!/bin/bash
WEB_ROOT=/var/www/crvfakeexample.com
CURRENTDATE=`date +"%Y%m%d-%H%M"`
export GIT_WORK_TREE=$WEB_ROOT/builds/$CURRENTDATE
export GIT_DIR=/var/www/crvfakeexample.git
export SYMFONY_ENV=prod
mkdir $GIT_WORK_TREE
git --work-tree=$GIT_WORK_TREE --git-dir=$GIT_DIR checkout -f master
rm -rf $GIT_WORK_TREE/var
ln -s $WEB_ROOT/shared/app/config/parameters.yml $GIT_WORK_TREE/app/config/parameters.yml
ln -sfn $WEB_ROOT/shared/var $GIT_WORK_TREE
cd $GIT_WORK_TREE
composer install --no-dev --optimize-autoloader
chown -R $(whoami):www-data $GIT_WORK_TREE
php bin/console cache:clear --env=prod --no-debug --no-warmup
php bin/console cache:warmup --env=prod
ln -sfn $GIT_WORK_TREE $WEB_ROOT/current
This script lives at /var/www/crvfakeexample.git/hooks/post-receive
for reference.
Let's break down the new additions:
CURRENTDATE=
date +"%Y%m%d-%H%M"``
We aren't creating a local build as we were when using rsync
. Instead, we need to instruct git on how to figure out the current time, and use that appropriately.
Thankfully we can make use of the built in Linux date
command, and some judicious use of switches can alter the output to meet our needs:
date +"%Y%m%d-%H%M"
- paste that into your terminal, and hit return. You should see something like: 20171118-2107
You may want to add in seconds, if you're the kind of person who deploys multiple times per minute. I'm not one of those people :)
We store the outcome of this command into the variable CURRENTDATE
.
export GIT_WORK_TREE=$WEB_ROOT/builds/$CURRENTDATE
We're still using the GIT_WORK_TREE
, but now we want to use a specific directory for each build, rather than a generic one which gets overwritten each time.
This won't work right off the bat. This directory must exist before we can start using it. This is why:
mkdir $GIT_WORK_TREE
We make sure that the directory exists before we go further.
Things are largely as before for the next few lines.
chown -R $(whoami):www-data $GIT_WORK_TREE
Permissions are such a pain. We have to get them right or things just won't work. But getting them right can be very fiddly.
Here we recursively change ownership of the GIT_WORK_TREE
directory (our current build) to which user we are connecting as (whichever user was set when we ran the git remote add prod ...
command from our local machine), and making sure the group www-data
has access too. This is very important as www-data
is the user our web server will run as, and that user is in the group www-data
.
Lastly we symlink the latest build to the $WEB_ROOT/current
directory.
Again, super important that we've updated our web site config accordingly, as we will now expect to be serving files from the current
ly symlinked directory.
Rollback and Cleanup
There are two further operations that we would need to be aware of:
Rollbacks are a manual operation.
Rolling back is as easy (or, as it may be, fiddly) as switching the symlink from builds/xyz
to builds/123
.
You need to make sure the build exists, and that your symlink command is correct, but with a bit of documentation (and practice) this isn't so bad. I mean, it's not great either, but as manual processes go, at least rolling back is feasible.
Cleanup could be automated.
Right now it's manual. The more deploys (git push
operations) we run, the more disk space we consume on the server. This can grow to be significant, as each build contains the full vendor
directory structure, which with our demo project is 73 megabytes (du -sh /var/www/crvfakeexample.com/current
).
We could augment our post-receive
script yet further to automatically delete the oldest directories greater than N:
cd $WEB_ROOT/builds
rm -rf $(ls -1t | tail -n +4)
(not my handywork, I stole this from here)
Note if using the above, +4
is misleading. It will keep only the most recent three directories. In other words, whatever number of builds you want to keep, bump it by one. Want to keep 10
, use +11
, etc.
Ok, that's about it for a git deploy.