Getting Comfortable with Symfony 4 Environment Variables

One of the biggest differences between a Symfony 2 or Symfony 3 deployment, and a Symfony 4 deployment is in the use of environment variables.

The Symfony deployment guide is really vague about the .env file at present.

Here's some extra info I found whilst reading the docs for the DotEnv component:

You should never store a .env file in your code repository as it might contain sensitive information; create a .env.dist file with sensible defaults instead.

Symfony Dotenv should only be used in development/testing/staging environments. For production environments, use "real" environment variables.

What the heck are "real" environment variables?

Different For You and Me

Here's the thing:

The advice needed here differs depending on your personal circumstances.

If you're on shared hosting, contact your host. Ask them how you can set environment variables. If they say you can't, you may need to switch hosting providers.

If you are on a VPS or dedicated, you should be ok. But there is a caveat:

The way I'm about to cover is a way. It is not the way, nor the only way.

You're going to need to use your own judgment, and adapt accordingly.

What Happens Currently?

Imagine we've rsync'd our data up the our server.

From the server, we try and run:

php bin/console # anything

PHP Fatal error:  Uncaught Error: Class 'Symfony\Component\Dotenv\Dotenv' not found in /var/www/crvfakeexample.com/bin/console:19
Stack trace:
#0 {main}
  thrown in /var/www/crvfakeexample.com/bin/console on line 19

Oh.

Oh dear.

What is line 19 trying to do?

// bin/console.php

if (!isset($_SERVER['APP_ENV'])) {
    (new Dotenv())->load(__DIR__.'/../.env');
}

It's looking for an environment variable: APP_ENV.

If it cannot find that environment variable it will fall back to the .env file in our project's root.

The Symfony docs are pretty explicit here, we shouldn't have an .env file in production.

Why are we making they making it so difficult for us!?

Given that we are getting the error above, it follows that APP_ENV must not be set.

For our local development it's not set either (at a system level), which is why Symfony falls back to loading the .env file contents.

This is good. It makes our lives easier. Most of the time.

We must now be aware of environment variables when going from dev to prod. And this means we need to remember that if we add a parameter to our application, we need to also ensure this .env file entry gets added to our production list of environment variables.

This may sound obvious, but I state this because on larger projects, for some clients, things take time. GitHub may deploy multiple times per day to production but your company may not be quite so bleeding edge. It may be that deploys take weeks between getting code finished, and that code being deployed to prod. This is exactly where you may forget about updating environment variables in the mean time.

Environment Variables

How to set up an environment variable then?

Again, there are multiple ways. And depending if we are talking Apache, or nginx, things differ further.

Let's begin by using Apache as our example web server.

Symfony's docs advise adding e.g. a SetEnv instruction to your Apache Vhost.

In doing this I've found the website will work, but the command line will break.

My Apache Vhost has no impact on running php from the command line. And nor should it.

We could add them here, but it would - potentially - duplicate our effort. DRY, and all that.

Instead, in my case, I use /etc/environment.

This is a simple file on the server that stores environment variables for interactive login sessions.

This likely won't work for shared hosting, to the very best of my knowledge.

Also, if you have multiple websites per server, this is going to need tweaking.

It's also important that you know:

• know this file exists
• keep it up to date
• keep it secure (hmmm)
• keep it backed up

Security here is a contentious subject.

By adding these entries to /etc/environment, we make our lives easier.

We also share this file's contents with any other user of our system.

There are trade offs at every corner.

And as mentioned, there are alternative approaches here. Choose what is right for you.

Adding Environment Variables

I'm going to add my environment variables to the file: /etc/environment.

The following command cat .env | grep ^[^#] should be run from your local development environment. This will look in your .env file and tell you exactly which lines are important, filtering any blank lines, or comments (#).

cat .env | grep ^[^#]

APP_ENV=dev
APP_DEBUG=1
APP_SECRET=67d829bf61dc5f87a73fd814e2c9f629
DATABASE_URL=mysql://root:pass@127.0.0.1:3306/symfony_demo
MAILER_URL=null://localhost

You need to copy these values over to your production server. And update accordingly.

# on your production server

# sudo vim /etc/environment

PATH="/whatever/you/have/on/your/path"
APP_ENV=prod
APP_DEBUG=0
APP_SECRET=some_new_secret_123
DATABASE_URL=mysql://not_root:different_pass@127.0.0.1:3306/symfony_demo
MAILER_URL=null://localhost

In the specific case of this example, we're not truly in production, so all we need to change is the APP_ENV and APP_DEBUG.

After doing this, save the file and exit.

You can type in env on your command line, and you get an dump of every set environment variable.

You won't see your new entries until you log out, and back in again, or you could also type source /etc/environment.

Now type env again and you should see your environment variables mixed in between everything that already existed.

Congratulations, the command line now works as expected:

php bin/console
Symfony 4.0.3 (kernel: src, env: prod, debug: false)

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -e, --env=ENV         The environment name [default: "prod"]
      --no-debug        Switches off debug mode
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  about                                   Displays information about the current project
  help                                    Displays help for a command
  list                                    Lists commands
 app
    ...

We can now install the demo application:

php bin/console doctrine:database:create
php bin/console doctrine:schema:create
php bin/console doctrine:fixtures:load

All should be good with our database, at this point.

However, the demo site will not work just yet.

If we hit the site, we see a blank page.

The var/log/prod.log may not even exist for you at this point.

Debugging this problem means looking at the Apache error logs, in our case:

tail -f crvfakeexample.com_error.log

[Fri Jan 12 14:29:56.977394 2018] [php7:error] [pid 4406] [client 192.168.0.44:57358] PHP Fatal error:  Uncaught Symfony\\Component\\Dotenv\\Exception\\PathException: Unable to read the "/var/www/crvfakeexample.com/public/../.env" environment file. in /var/www/crvfakeexample.com/vendor/symfony/dotenv/Dotenv.php:54\nStack trace:\n#0 /var/www/crvfakeexample.com/public/index.php(21): Symfony\\Component\\Dotenv\\Dotenv->load('/var/www/crvfak...')\n#1 {main}\n  thrown in /var/www/crvfakeexample.com/vendor/symfony/dotenv/Dotenv.php on line 54

It's not pretty, but it's helpful.

Line 21 of public/index.php is the cause of our issue:

// public/index.php

// The check is to ensure we don't use .env in production
if (!isset($_SERVER['APP_ENV'])) {
    (new Dotenv())->load(__DIR__.'/../.env');
}

Even though we have set the environment variables, and in particular the APP_ENV environment variable, it's not being passed through for some reason.

What we need to do is to tell our web site about the APP_ENV environment variable, explicitly.

As a heads up: There may be a more elegant solution to this problem. This following does work for me:

Start By Just Defining One Variable

We need to get - at least - the APP_ENV environment variable into our web site config. In my case this is an Apache Vhost, so what I need to do is follow the guidance from the official web server configuration docs:

# /etc/apache2/sites-available/crvfakeexample.com.conf

    # optionally set the value of the environment variables used in the application
    SetEnv APP_ENV prod
    #SetEnv APP_SECRET <app-secret-id>
    #SetEnv DATABASE_URL "mysql://db_user:db_pass@host:3306/db_name"
</VirtualHost>

I've removed the comment from the line SetEnv APP_ENV prod.

This hardcodes my Apache Vhost to be always in prod.

Remember:

sudo service apache2 reload

If we try to visit the site now, it still fails but the error has "improved":

[Fri Jan 12 14:37:59.835908 2018] [php7:error] [pid 4547] [client 192.168.0.44:57512] PHP Fatal error:  Uncaught Symfony\\Component\\DependencyInjection\\Exception\\EnvNotFoundException: Environment variable not found: "APP_SECRET". in /var/www/crvfakeexample.com/vendor/symfony/dependency-injection/EnvVarProcessor.php:76\nStack trace:\n#0 /var/www/crvfakeexample.com/vendor/symfony/dependency-injection/Container.php(387): Symfony\\Component\\DependencyInjection\\EnvVarProcessor->getEnv('string', 'APP_SECRET', Object(Closure))\n#1 /var/www/crvfakeexample.com/var/cache/prod/ContainerSbWz5V9/srcProdProjectContainer.php(733): Symfony\\Component\\DependencyInjection\\Container->getEnv('APP_SECRET')\n#2 /var/www/crvfakeexample.com/var/cache/prod/ContainerSbWz5V9/srcProdProjectContainer.php(383): ContainerSbWz5V9\\srcProdProjectContainer->getFragment_ListenerService()\n#3 /var/www/crvfakeexample.com/vendor/symfony/event-dispatcher/EventDispatcher.php(229): ContainerSbWz5V9\\srcProdProjectContainer->ContainerSbWz5V9\\{closure}()\n#4 /var/www/crvfakeexample.com/vendor/symfony/event-dispatcher/EventDispatcher.php(61): Symfony\\Component\\Even in /var/www/crvfakeexample.com/templates/base.html.twig on line 104

The key part: 'Environment variable not found: "APP_SECRET"'

Ok, so we can go back into the Apache Vhost and fix this, right?

# /etc/apache2/sites-available/crvfakeexample.com.conf

    # optionally set the value of the environment variables used in the application
    SetEnv APP_ENV prod
    SetEnv APP_SECRET whatever_we_need_here
    SetEnv DATABASE_URL "mysql://prod_user:some_long_complex_password@our_db_host:3306/db_name"
</VirtualHost>

And this works.

We'd also need to go and add a SetEnv line for every environment variable in our project.

Why bother setting them in the /etc/environment file then?

Because if we don't, then the cli / command line interface (e.g. php bin/console) won't work for us.

Eliminate Duplication

If we need to have all these entries in our Apache Vhost, let's not duplicate them.

We can instead use the environment variable values dynamically:

# /etc/apache2/sites-available/crvfakeexample.com.conf

    # optionally set the value of the environment variables used in the application
    SetEnv APP_ENV ${APP_ENV}
    SetEnv APP_DEBUG ${APP_DEBUG}
    SetEnv APP_SECRET ${APP_SECRET}
    SetEnv DATABASE_URL ${DATABASE_URL}
    SetEnv MAILER_URL ${MAILER_URL}
</VirtualHost>

This won't work immediately.

We need to update /etc/environment slightly:

# /etc/environment

PATH="/whatever/you/have/on/your/path"
export APP_ENV=prod
export APP_DEBUG=0
export APP_SECRET=some_new_secret_123
export DATABASE_URL=mysql://not_root:different_pass@127.0.0.1:3306/symfony_demo
export MAILER_URL=null://localhost

We need to explicitly tell Apache to load these extra configured environment variables:

sudo vim /etc/apache2/envvars

And at the bottom of this file, add the line:

. /etc/environment

This ensures Apache loads our preset environment variables. If we miss this step, Apache is none the wiser.

Again, restart Apache:

sudo service apache2 restart

And that should be good enough to satisfy Apache.

nginx, however, isn't quite so ... understanding.

Environment Variables and nginx

To the very best of my knowledge, you cannot use environment variables dynamically inside an nginx server configuration file. Not without extra plugins (Lua), at least.

Unfortunately this means we cannot remain DRY.

We must repeat ourselves.

This means we must store the environment variables inside the nginx server configuration for our particular website, AND as system wide environment variables inside /etc/environment.

It feels like there should be a nicer solution to this. Personally I mitigate this, somewhat, by using Docker.

Instead, in this case we must hardcore the extra config into our nginx server definition:

server {
    listen 80 default;
    server_name crvfakeexample.com www.crvfakeexample.com;
    root /var/www/crvfakeexample.com/public;

    location / {
        # try to serve file directly, fallback to index.php
        try_files $uri /index.php$is_args$args;
    }

    location ~ ^/index\.php(/|$) {
        fastcgi_pass unix:/run/php/php7.2-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;

        # optionally set the value of the environment variables used in the application
+       fastcgi_param APP_ENV prod;
+       fastcgi_param APP_SECRET some_new_secret_123;
+       fastcgi_param APP_DEBUG 0;
+       fastcgi_param DATABASE_URL "mysql://not_root:different_pass@127.0.0.1:3306/symfony_demo";
+       fastcgi_param MAILER_URL null://localhost;

        # When you are using symlinks to link the document root to the
        # current version of your application, you should pass the real
        # application path instead of the path to the symlink to PHP
        # FPM.
        # Otherwise, PHP's OPcache may not properly detect changes to
        # your PHP files (see https://github.com/zendtech/ZendOptimizerPlus/issues/126
        # for more information).
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $realpath_root;
        # Prevents URIs that include the front controller. This will 404:
        # http://domain.tld/index.php/some-path
        # Remove the internal directive to allow URIs like this
        internal;
    }

    # return 404 for all other php files not matching the front controller
    # this prevents access to other php files you don't want to be accessible.
    location ~ \.php$ {
        return 404;
    }

    error_log /var/log/nginx/crvfakeexample.com_error.log;
    access_log /var/log/nginx/crvfakeexample.com_access.log;
}

This should be good enough to get you up and running with nginx.