Part 3 / 3 - Deploying Symfony 4 with git - Adding Builds


Previously we setup our Git Deployment for Symfony 4 to git push our local changes up (and over the top of) any existing live code on our website.

This comes with a big warning alarm that you may break the live site for real visitors whilst you are doing your deploy.

A better approach would be to do use the concept of a build directory, and use a symlink / shortcut to point Apache / nginx at your desired build.

If you've followed either the rsync approach for Symfony 2/3, or the Symfony 4 variant, you will have an idea as to what we're going for here.

What we're about to do is extremely similar to what we did for Symfony 2 / Symfony 3. That video / write up contains a bit more information than we will go into here, for reasons of not repeating myself.

Out With The Code, In With The Builds

Much like previously, I'm going to assume that you are working in a controlled demonstration environment, and that the simplest way to 'reset' is to delete everything and start from fresh.

If you are working in the real world, take the appropriate precautions.

# from the server

cd /var/www/crvfakeexample.com
rm -rf code/
mkdir builds

I am working on the assumption that you have completed the steps from the previous video on sharing the var directory.

Whereas when we implemented a similar approach using rsync we had to manually 'timestamp' our builds, in this approach we will get git to do this dirty work for us.

Let's amend the post-receive hook file to take care of this for us:

# from the server

sudo vim /var/www/crvfakeexample.git/hooks/post-receive

Here's the update in full:

#!/bin/bash

WEB_ROOT=/var/www/crvfakeexample.com
CURRENTDATE=`date +"%Y%m%d-%H%M"`

export GIT_WORK_TREE=$WEB_ROOT/builds/$CURRENTDATE
export GIT_DIR=/var/www/crvfakeexample.git

mkdir $GIT_WORK_TREE

git --work-tree=$GIT_WORK_TREE --git-dir=$GIT_DIR checkout -f master

rm -rf $GIT_WORK_TREE/var
ln -sfn $WEB_ROOT/shared/var $GIT_WORK_TREE

cd $GIT_WORK_TREE

composer install --no-dev --optimize-autoloader

chown -R $(whoami):www-data $GIT_WORK_TREE

php bin/console cache:clear --no-warmup
php bin/console cache:warmup

ln -sfn $GIT_WORK_TREE $WEB_ROOT/current

All of these changes shown here are described already in this video. Please watch / read for a full explanation if anything is unsure, or ask in the comments section below.

At this point we should be able to deploy.

Again, if using the same project as you have been in the previous two videos, you will need to fake a change through here:

# from your local

cd {project dir}
touch some-change
git add some-change
git commit -m "another fake change"

git push prod master

# output here

But wait, we hit on an error:

gp prod master

# asked for password here

Counting objects: 2, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 241 bytes | 0 bytes/s, done.
Total 2 (delta 1), reused 0 (delta 0)
remote: Already on 'master'
remote: Loading composer repositories with package information
remote: Installing dependencies from lock file
remote: Package operations: 70 installs, 0 updates, 0 removals
remote:   - Installing ocramius/package-versions (1.2.0): Loading from cache
remote:   - Installing symfony/flex (v1.0.60): Loading from cache
remote:   - Installing symfony/polyfill-mbstring (v1.6.0): Loading from cache
remote:   - Installing doctrine/lexer (v1.0.1): Loading from cache
remote:   - Installing doctrine/inflector (v1.3.0): Loading from cache
remote:   - Installing doctrine/collections (v1.5.0): Loading from cache
remote:   - Installing doctrine/cache (v1.7.1): Loading from cache
remote:   - Installing doctrine/annotations (v1.6.0): Loading from cache
remote:   - Installing doctrine/common (v2.8.1): Loading from cache
remote:   - Installing symfony/doctrine-bridge (v4.0.3): Loading from cache
remote:   - Installing doctrine/doctrine-cache-bundle (1.3.2): Loading from cache
remote:   - Installing symfony/routing (v4.0.3): Loading from cache
remote:   - Installing symfony/http-foundation (v4.0.3): Loading from cache
remote:   - Installing symfony/event-dispatcher (v4.0.3): Loading from cache
remote:   - Installing psr/log (1.0.2): Loading from cache
remote:   - Installing symfony/debug (v4.0.3): Loading from cache
remote:   - Installing symfony/http-kernel (v4.0.3): Loading from cache
remote:   - Installing symfony/finder (v4.0.3): Loading from cache
remote:   - Installing symfony/filesystem (v4.0.3): Loading from cache
remote:   - Installing psr/container (1.0.0): Loading from cache
remote:   - Installing symfony/dependency-injection (v4.0.3): Loading from cache
remote:   - Installing symfony/config (v4.0.3): Loading from cache
remote:   - Installing psr/simple-cache (1.0.0): Loading from cache
remote:   - Installing psr/cache (1.0.1): Loading from cache
remote:   - Installing symfony/cache (v4.0.3): Loading from cache
remote:   - Installing symfony/framework-bundle (v4.0.3): Loading from cache
remote:   - Installing symfony/console (v4.0.3): Loading from cache
remote:   - Installing jdorn/sql-formatter (v1.2.17): Loading from cache
remote:   - Installing doctrine/dbal (v2.6.3): Loading from cache
remote:   - Installing doctrine/doctrine-bundle (1.8.1): Loading from cache
remote:   - Installing doctrine/data-fixtures (v1.3.0): Loading from cache
remote:   - Installing doctrine/doctrine-fixtures-bundle (3.0.2): Loading from cache
remote:   - Installing symfony/yaml (v4.0.3): Loading from cache
remote:   - Installing zendframework/zend-eventmanager (3.2.0): Loading from cache
remote:   - Installing zendframework/zend-code (3.3.0): Loading from cache
remote:   - Installing ocramius/proxy-manager (2.1.1): Loading from cache
remote:   - Installing doctrine/migrations (v1.6.2): Loading from cache
remote:   - Installing doctrine/doctrine-migrations-bundle (v1.3.1): Loading from cache
remote:   - Installing doctrine/instantiator (1.1.0): Loading from cache
remote:   - Installing doctrine/orm (v2.6.0): Loading from cache
remote:   - Installing egulias/email-validator (2.1.3): Loading from cache
remote:   - Installing erusev/parsedown (1.6.4): Loading from cache
remote:   - Installing ezyang/htmlpurifier (v4.9.3): Loading from cache
remote:   - Installing sensio/framework-extra-bundle (v5.1.3): Loading from cache
remote:   - Installing composer/ca-bundle (1.1.0): Loading from cache
remote:   - Installing sensiolabs/security-checker (v4.1.7): Loading from cache
remote:   - Installing symfony/asset (v4.0.3): Loading from cache
remote:   - Installing symfony/expression-language (v4.0.3): Loading from cache
remote:   - Installing symfony/inflector (v4.0.3): Loading from cache
remote:   - Installing symfony/property-access (v4.0.3): Loading from cache
remote:   - Installing symfony/options-resolver (v4.0.3): Loading from cache
remote:   - Installing symfony/intl (v4.0.3): Loading from cache
remote:   - Installing symfony/polyfill-intl-icu (v1.6.0): Loading from cache
remote:   - Installing symfony/form (v4.0.3): Loading from cache
remote:   - Installing monolog/monolog (1.23.0): Loading from cache
remote:   - Installing symfony/monolog-bridge (v4.0.3): Loading from cache
remote:   - Installing symfony/monolog-bundle (v3.1.2): Loading from cache
remote:   - Installing symfony/polyfill-apcu (v1.6.0): Loading from cache
remote:   - Installing symfony/security (v4.0.3): Loading from cache
remote:   - Installing symfony/security-bundle (v4.0.3): Loading from cache
remote:   - Installing swiftmailer/swiftmailer (v6.0.2): Loading from cache
remote:   - Installing symfony/swiftmailer-bundle (v3.1.6): Loading from cache
remote:   - Installing twig/twig (v2.4.4): Loading from cache
remote:   - Installing symfony/twig-bridge (v4.0.3): Loading from cache
remote:   - Installing symfony/translation (v4.0.3): Loading from cache
remote:   - Installing symfony/validator (v4.0.3): Loading from cache
remote:   - Installing twig/extensions (v1.5.1): Loading from cache
remote:   - Installing symfony/twig-bundle (v4.0.3): Loading from cache
remote:   - Installing pagerfanta/pagerfanta (v1.0.5): Loading from cache
remote:   - Installing white-october/pagerfanta-bundle (v1.1.2): Loading from cache
remote: Generating optimized autoload files
remote: ocramius/package-versions:  Generating version class...
remote: ocramius/package-versions: ...done generating version class
remote: PHP Fatal error:  Cannot declare interface Symfony\Component\HttpKernel\HttpKernelInterface, because the name is already in use in /var/www/crvfakeexample.com/builds/20180116-1505/vendor/symfony/http-kernel/HttpKernelInterface.php on line 22
remote: PHP Fatal error:  Cannot declare interface Symfony\Component\HttpKernel\HttpKernelInterface, because the name is already in use in /var/www/crvfakeexample.com/builds/20180116-1505/vendor/symfony/http-kernel/HttpKernelInterface.php on line 22
To root@104.236.215.199:/var/www/crvfakeexample.git

Hmm.

And if we go to the server and try to do things:

php bin/console cache:clear

PHP Fatal error:  Cannot declare interface Symfony\Component\HttpKernel\HttpKernelInterface, because the name is already in use in /var/www/crvfakeexample.com/builds/20180116-1505/vendor/symfony/http-kernel/HttpKernelInterface.php on line 22

Oh my.

This issue is the only thing I can find on this front, and it's quite old now, and not related to Symfony 4. Also the fix is merged, so in theory, we shouldn't be seeing this.

Only we are.

Ok, so here's my fix:

# from the server

rm -rf /var/www/crvfakeexample.com/current/var/cache/prod/

This fixes the problem.

php bin/console cache:warmup

 // Warming up the cache for the prod environment with debug false                                                      

 [OK] Cache for the "prod" environment (debug=false) was successfully warmed.                                           

Let's integrate this into our post-receive hook file:

#!/bin/bash

WEB_ROOT=/var/www/crvfakeexample.com
CURRENTDATE=`date +"%Y%m%d-%H%M"`

export GIT_WORK_TREE=$WEB_ROOT/builds/$CURRENTDATE
export GIT_DIR=/var/www/crvfakeexample.git

mkdir $GIT_WORK_TREE

git --work-tree=$GIT_WORK_TREE --git-dir=$GIT_DIR checkout -f master

rm -rf $GIT_WORK_TREE/var
ln -sfn $WEB_ROOT/shared/var $GIT_WORK_TREE

cd $GIT_WORK_TREE

+ rm -rf ./var/cache/prod

composer install --no-dev --optimize-autoloader

chown -R $(whoami):www-data $GIT_WORK_TREE

php bin/console cache:clear --no-warmup
php bin/console cache:warmup

ln -sfn $GIT_WORK_TREE $WEB_ROOT/current

And that seems to fix the issue.

Just a side note here, there's nothing untoward about deleting this cache/prod directory. The original Symfony 4 plan was to include a Makefile, and one of the entries of that Makefile was:

cache-clear:
    @test -f bin/console && bin/console cache:clear --no-warmup || rm -rf var/cache/*

If problems persist from this point it is worth re-applying the var directory permissions:

cd /var/www/crvfakeexample.com/shared

rm -rf ./var
mkdir var
sudo chown -R www-data:www-data var

HTTPDUSER=$(ps axo user,comm | grep -E '[a]pache|[h]ttpd|[_]www|[w]ww-data|[n]ginx' | grep -v root | head -1 | cut -d\  -f1)
sudo setfacl -dR -m u:"$HTTPDUSER":rwX -m u:$(whoami):rwX var
sudo setfacl -R -m u:"$HTTPDUSER":rwX -m u:$(whoami):rwX var

This is a more drastic step, but resolved my issues.

Web Server Config

In order to make this change work as expected, we need to update the Apache / nginx site configs to use .../current in their paths:

Revised Apache Vhost Configuration

<VirtualHost *:80>
    ServerName crvfakeexample.com
    ServerAlias www.crvfakeexample.com

    DocumentRoot /var/www/crvfakeexample.com/current/public
    <Directory /var/www/crvfakeexample.com/current/public>
        AllowOverride None
        Require all granted
        Allow from All

        <IfModule mod_rewrite.c>
            Options -MultiViews
            RewriteEngine On
            RewriteCond %{REQUEST_FILENAME} !-f
            RewriteRule ^(.*)$ index.php [QSA,L]
        </IfModule>
    </Directory>

    # uncomment the following lines if you install assets as symlinks
    # or run into problems when compiling LESS/Sass/CoffeeScript assets
    # <Directory /var/www/crvfakeexample.com/code>
    #     Options FollowSymlinks
    # </Directory>

    # optionally disable the RewriteEngine for the asset directories
    # which will allow apache to simply reply with a 404 when files are
    # not found instead of passing the request into the full symfony stack
    <Directory /var/www/crvfakeexample.com/current/public/bundles>
        <IfModule mod_rewrite.c>
            RewriteEngine Off
        </IfModule>
    </Directory>
    ErrorLog /var/log/apache2/crvfakeexample.com_error.log
    CustomLog /var/log/apache2/crvfakeexample.com_access.log combined

    # optionally set the value of the environment variables used in the application
    SetEnv APP_ENV ${APP_ENV}
    SetEnv APP_DEBUG ${APP_DEBUG}
    SetEnv APP_SECRET ${APP_SECRET}
    SetEnv DATABASE_URL ${DATABASE_URL}
</VirtualHost>

Be sure to run an Apache reload after changing this config:

service apache2 reload

Revised nginx Server Configuration

server {
    listen 80 default;
    server_name crvfakeexample.com www.crvfakeexample.com;
    root /var/www/crvfakeexample.com/current/public;

    location / {
        # try to serve file directly, fallback to index.php
        try_files $uri /index.php$is_args$args;
    }

    location ~ ^/index\.php(/|$) {
        fastcgi_pass unix:/run/php/php7.2-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;

        # optionally set the value of the environment variables used in the application
        fastcgi_param APP_ENV prod;
        fastcgi_param APP_SECRET some_new_secret_123;
        fastcgi_param APP_DEBUG 0;
        fastcgi_param DATABASE_URL "mysql://not_root:different_pass@127.0.0.1:3306/symfony_demo";
        fastcgi_param MAILER_URL null://localhost;

        # When you are using symlinks to link the document root to the
        # current version of your application, you should pass the real
        # application path instead of the path to the symlink to PHP
        # FPM.
        # Otherwise, PHP's OPcache may not properly detect changes to
        # your PHP files (see https://github.com/zendtech/ZendOptimizerPlus/issues/126
        # for more information).
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $realpath_root;
        # Prevents URIs that include the front controller. This will 404:
        # http://domain.tld/index.php/some-path
        # Remove the internal directive to allow URIs like this
        internal;
    }

    # return 404 for all other php files not matching the front controller
    # this prevents access to other php files you don't want to be accessible.
    location ~ \.php$ {
        return 404;
    }

    error_log /var/log/nginx/crvfakeexample.com_error.log;
    access_log /var/log/nginx/crvfakeexample.com_access.log;
}

Be sure to run an nginx reload after changing this config:

service nginx configtest
service nginx reload

That's enough to get us deployed and up and running.

There is a little extra we could do around automating house keeping, but for the most part, we are done.

Episodes