Part 2/3 - Deploying with git - Sharing folders


In the previous video we covered how to deploy a Symfony site using a git push combined with a post-receive hook.

As I mentioned in that video, deploying with git is really not something I would pursue. Other better options are available.

However, if I were to pursue this approach, there are some tweaks I would make to the previous setup.

Much like when we looked at deploying with rsync, it would be nicer (in my opinion) to extract some of the 'common' files and folders out from the root of the project, and into a shared directory.

Please note that this is not required.

We can / could make edits directly to the files / folders created and managed by git, and maybe we wouldn't have any issues. One such example of this would be in our parameters.yml file, which we saw that our post-receive's call to composer install ... will create for us.

Because parameters.yml is essentially outside the remit of git, we could make changes to the file directly on the server, and never, to the best of my knowledge, need to worry about losing those changes.

Likewise, because we are not changing the underlying directory as we did when switching the current symlink in our rsync deploy, we need not worry about extracting the var directory.

However, I am going to suggest we extract both anyway.

Why?

Because it makes it very obvious that these files and folders are special.

But more to the point, it makes it that much harder for us, or one of our team to inadvertently break things.

Extracting And Sharing

This change will require a modification to our directory structure, which means we will need to update the post-receive hook file, and Apache / nginx site config accordingly.

Here's what we have currently:

# server side

cd /var/www/crvfakeexample.com

ls -la

total 376
drwxr-xr-x  10 root  www-data   4096 Nov 18 12:20 .
drwxr-xr-x   5 root  root       4096 Nov 18 11:07 ..
drwxrwxr-x   4 root  root      4096 Nov 18 12:03 app
-rw-rw-r--   1 root  root      1357 Nov 18 12:03 app.json
-rw-rw-r--   1 root  root      2166 Nov 18 12:03 appveyor.yml
drwxrwxr-x   4 root  root      4096 Nov 18 12:03 assets
drwxrwxr-x   2 root  root      4096 Nov 18 12:20 bin
-rw-rw-r--   1 root  root      2991 Nov 18 12:03 composer.json
-rw-rw-r--   1 root  root    117317 Nov 18 12:03 composer.lock
-rw-rw-r--   1 root  root       368 Nov 18 12:03 CONTRIBUTING.md
-rw-rw-r--   1 root  root       131 Nov 18 11:57 .editorconfig
-rw-rw-r--   1 root  root       352 Nov 18 11:57 .gitignore
-rw-rw-r--   1 root  root      1065 Nov 18 12:03 LICENSE
-rw-rw-r--   1 root  root       715 Nov 18 12:03 package.json
-rw-rw-r--   1 root  root      1300 Nov 18 11:57 .php_cs.dist
-rw-rw-r--   1 root  root      1294 Nov 18 12:03 phpunit.xml.dist
-rw-rw-r--   1 root  root        65 Nov 18 12:03 README.md
-rw-rw-r--   1 root  root         0 Nov 18 12:06 some-file
drwxrwxr-x   4 root  root      4096 Nov 18 12:03 src
drwxrwxr-x   3 root  root      4096 Nov 18 12:03 tests
-rw-rw-r--   1 root  root      1042 Nov 18 11:57 .travis.yml
drwxrwxr-x+  6 root  www-data  4096 Nov 18 12:20 var
drwxrwxr-x  20 root  root      4096 Nov 18 12:20 vendor
drwxrwxr-x   4 root  root      4096 Nov 18 12:20 web
-rw-rw-r--   1 root  root       825 Nov 18 12:03 webpack.config.js
-rw-rw-r--   1 root  root    167005 Nov 18 12:03 yarn.lock

Our new directory structure will be:

  • /var/www/crvfakeexample.com/shared/app/config/parameters.yml
  • /var/www/crvfakeexample.com/shared/var
  • /var/www/crvfakeexample.com/code

Feel free to name things differently, of course.

code will contain everything currently in /var/www/crvfakeexample.com/.

We'll need to add in symlinks via the post-receive hook.

Under Construction

If your site is live then this process needs a little more forethought.

As our site is purely for demo purposes, the easy approach is to delete everything from /var/www/crvfakeexample.com and start again.

# server side

rm -rf /var/www/crvfakeexample.com/*
rm -rf /var/www/crvfakeexample.com/.*

The second command removes any hidden files that are left over. There might be a one line approach to this, so if you know it, please share.

Next, create the new directory structure:

mkdir code
mkdir -p shared/app/config
mkdir shared/var

And before we go further, be sure to sort out those pesky Symfony permissions problems before they can bite us:

HTTPDUSER=$(ps axo user,comm | grep -E '[a]pache|[h]ttpd|[_]www|[w]ww-data|[n]ginx' | grep -v root | head -1 | cut -d\  -f1)
sudo setfacl -dR -m u:"$HTTPDUSER":rwX -m u:$(whoami):rwX var
sudo setfacl -R -m u:"$HTTPDUSER":rwX -m u:$(whoami):rwX var

ls -la /var/www/crvfakeexample.com/shared

total 20
drwxrwxr-x  4 root  root     4096 Nov 18 12:34 .
drwxr-xr-x  4 root  www-data 4096 Nov 18 12:33 ..
drwxrwxr-x  3 root  root     4096 Nov 18 12:34 app
drwxrwxr-x+ 2 root  root     4096 Nov 18 12:34 var

I'm going to set ownership of everything in the shared directory to www-data:www-data:

sudo chown -R www-data:www-data /var/www/crvfakeexample.com/shared

ls -la /var/www/crvfakeexample.com/shared

total 20
drwxrwxr-x  4 www-data www-data 4096 Nov 18 12:34 .
drwxr-xr-x  4 root     www-data 4096 Nov 18 12:33 ..
drwxrwxr-x  3 www-data www-data 4096 Nov 18 12:34 app
drwxrwxr-x+ 2 www-data www-data 4096 Nov 18 12:34 var

This should have preempted any permissions problems we might have otherwise faced.

Setting up parameters.yml

Next we need to add in the expected values in to parameters.yml. Remember we do this on the server side, and these values are those needed to make our production config work:

# /var/www/crvfakeexample.com/shared/app/config/parameters.yml

parameters:
    locale: en
    env(SYMFONY_SECRET): secret_value_for_symfony_demo_application
    env(LOG_URL): '%kernel.logs_dir%/%kernel.environment%.log'
    env(DATABASE_URL): 'mysql://root:pass@127.0.0.1:3306/symfony_demo'
    env(MAILER_URL): 'smtp://localhost:25?encryption=&auth_mode='

In our case these are identical values to those from parameters.yml.dist, but they very likely would not be in true production. Now we can update them without mucking around directly in the production directory.

We're going to need to symlink parameters.yml into place, and we will get to that shortly.

Updating post-receive

Next we need to update our post-receive hook file.

Again from the server:

vim /var/www/crvfakeexample.git/hooks/post-receive

First we want to update the GIT_WORK_TREE environment variable:

#!/bin/bash

export GIT_WORK_TREE=/var/www/crvfakeexample.com/code

# ...

Next we need to use symlinks for var and app/config/parameters.yml.

What I don't want is a bunch of repeated paths, particularly the base part of our root directory (/var/www/crvfakeexample.com).

Let's extract this out to its own variable, and use that instead:

#!/bin/bash

WEB_ROOT=/var/www/crvfakeexample.com

export GIT_WORK_TREE=$WEB_ROOT/code

# ...

Which means we can then use the WEB_ROOT variable for our symlink addition:

#!/bin/bash

WEB_ROOT=/var/www/crvfakeexample.com

export GIT_WORK_TREE=$WEB_ROOT/code

# ...

rm -rf $GIT_WORK_TREE/var

ln -s $WEB_ROOT/shared/app/config/parameters.yml $GIT_WORK_TREE/app/config/parameters.yml
ln -sfn $WEB_ROOT/shared/var $GIT_WORK_TREE

# ...

We will start by removing the existing var directory. If we don't do this then the symlink operation will fail as the var directory will already exist.

Admittedly the syntax for the second symlink looks kind of strange.

Putting everything together gives us:

#!/bin/bash

WEB_ROOT=/var/www/crvfakeexample.com

export GIT_WORK_TREE=$WEB_ROOT/code
export GIT_DIR=/var/www/crvfakeexample.git
export SYMFONY_ENV=prod

git --work-tree=$GIT_WORK_TREE --git-dir=$GIT_DIR checkout -f master

rm -rf $GIT_WORK_TREE/var

ln -s $WEB_ROOT/shared/app/config/parameters.yml $GIT_WORK_TREE/app/config/parameters.yml
ln -sfn $WEB_ROOT/shared/var $GIT_WORK_TREE

cd $GIT_WORK_TREE

composer install --no-dev --optimize-autoloader

php bin/console cache:clear --env=prod --no-debug --no-warmup
php bin/console cache:warmup --env=prod

You could add in echo statements to give progress updates, or visual feedback, or whatever you like.

Doing a git push prod master now should see us right. But as we've changed the directory structure we do need to tell Apache / nginx.

Updating Web Server Config

Before we can see our changes we need to make sure the web server config is pointed at our revised root directory.

For Apache:

<VirtualHost *:80>
    ServerName crvfakeexample.com
    ServerAlias www.crvfakeexample.com

    DocumentRoot /var/www/crvfakeexample.com/code/web
    <Directory /var/www/crvfakeexample.com/code/web>
        AllowOverride None
        Order Allow,Deny
        Allow from All

        <IfModule mod_rewrite.c>
            Options -MultiViews
            RewriteEngine On
            RewriteCond %{REQUEST_FILENAME} !-f
            RewriteRule ^(.*)$ app.php [QSA,L]
        </IfModule>
    </Directory>

    # uncomment the following lines if you install assets as symlinks
    # or run into problems when compiling LESS/Sass/CoffeeScript assets
    # <Directory /var/www/crvfakeexample.com/code>
    #     Options FollowSymlinks
    # </Directory>

    # optionally disable the RewriteEngine for the asset directories
    # which will allow apache to simply reply with a 404 when files are
    # not found instead of passing the request into the full symfony stack
    <Directory /var/www/crvfakeexample.com/code/web/bundles>
        <IfModule mod_rewrite.c>
            RewriteEngine Off
        </IfModule>
    </Directory>
    ErrorLog /var/log/apache2/crvfakeexample_com_error.log
    CustomLog /var/log/apache2/crvfakeexample_com_access.log combined
</VirtualHost>

Don't forget to reload Apache:

sudo service apache reload

And for nginx:

server {
    listen 80 default;
    server_name crvfakeexample.com www.crvfakeexample.com;
    root /var/www/crvfakeexample.com/code/web;

    location / {
        # try to serve file directly, fallback to app.php
        try_files $uri /app.php$is_args$args;
    }

    # DEV
    # This rule should only be placed on your development environment
    # In production, don't include this and don't deploy app_dev.php or config.php
    location ~ ^/(app_.*|config)\.php(/|$) {
        fastcgi_pass unix:/run/php/php7.0-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $realpath_root;
    }

    # PROD
    location ~ ^/app\.php(/|$) {
        fastcgi_pass unix:/run/php/php7.0-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $realpath_root;
        internal;
    }

    # return 404 for all other php files not matching the front controller
    # this prevents access to other php files you don't want to be accessible.
    location ~ \.php$ {
      return 404;
    }

    error_log /var/log/nginx/crvfakeexample.com_error.log;
    access_log /var/log/nginx/crvfakeexample.com_access.log;
}

Don't forget to reload nginx:

sudo service nginx configtest
sudo service nginx reload

Basically a find / replace exercise.

Testing Our Revised Git Deployment

Ok, let's push:

git push prod master

# password prompt here

Everything up-to-date

Oh my, how anti-climactic.

Let's fake a change through:

touch fake-change

git add fake-change
git commit -m "a fake change"

[master 3a5c40b] a fake change
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 fake-change

git push prod master

# output here

This business of having to 'fake a change' to do a deploy is part of the reason I don't like using git like this.

One of the benefits of this revised approach is that you can now safely completely delete the contents of /var/www/crvfakeexample.com/code without worry of losing any customisations to our application config. Any custom changes would be better kept outside the live code directory as shown here, in my opinion.

Episodes