Docker Elixir Phoenix - Part 1 - The Web Server


In the previous video we made a generic image containing the core functionality needed to run an instance of the Phoenix framework.

We put all of this into a Dockerfile, which we could build and push up to Docker Hub. To make our lives easier we made a Makefile containing shortcuts to build and push.

The reasoning for doing all of this is to remove the need to build an image every time we want to do a spot of development work.

Where things get more interesting is in spinning up an instance of the Phoenix framework and seeing it in our browser.

To make this as easy as possible we will use a docker-compose.yml file to define our infrastructure, and a few choice Makefile commands to get us up and running quickly. You don't need to use the Makefile, it just makes life easier, excuse the pun.

In the previous video we worked out of the docker-phoenix directory.

In this video we will create a new directory, one that will hold our development instance of the Phoenix Framework code. To do this, we will need to re-run through the Phoenix Framework setup, but this time on our local computer. This is not the same as the work we did inside the Docker image.

Please follow the instructions for installing Elixir, Erlang, and the Phoenix dependencies on to your local machine, if you have no already done so.

All set? Good, on we go!

Let's imagine we are working from our local PC.

We have a directory where we do all our dev work:

cd ~/Development
ls
{other stuff}
docker-phoenix

In other words, we want to keep the Phoenix Docker setup completely independent to our particular project. This means we only need to set up our Phoenix server image once, and then we can use it in as many Phoenix projects as we like. Very cool.

Let's now create a new directory using the mix phx.new command:

mix phx.new hello_phoenix

* creating hello_phoenix/config/config.exs
* creating hello_phoenix/config/dev.exs
* creating hello_phoenix/config/prod.exs
* creating hello_phoenix/config/prod.secret.exs
* creating hello_phoenix/config/test.exs
* creating hello_phoenix/lib/hello_phoenix/application.ex
* creating hello_phoenix/lib/hello_phoenix.ex
* creating hello_phoenix/lib/hello_phoenix_web/channels/user_socket.ex
* creating hello_phoenix/lib/hello_phoenix_web/views/error_helpers.ex
* creating hello_phoenix/lib/hello_phoenix_web/views/error_view.ex
* creating hello_phoenix/lib/hello_phoenix_web/endpoint.ex
* creating hello_phoenix/lib/hello_phoenix_web/router.ex
* creating hello_phoenix/lib/hello_phoenix_web.ex
* creating hello_phoenix/mix.exs
* creating hello_phoenix/README.md
* creating hello_phoenix/test/support/channel_case.ex
* creating hello_phoenix/test/support/conn_case.ex
* creating hello_phoenix/test/test_helper.exs
* creating hello_phoenix/test/hello_phoenix_web/views/error_view_test.exs
* creating hello_phoenix/lib/hello_phoenix_web/gettext.ex
* creating hello_phoenix/priv/gettext/en/LC_MESSAGES/errors.po
* creating hello_phoenix/priv/gettext/errors.pot
* creating hello_phoenix/lib/hello_phoenix/repo.ex
* creating hello_phoenix/priv/repo/seeds.exs
* creating hello_phoenix/test/support/data_case.ex
* creating hello_phoenix/lib/hello_phoenix_web/controllers/page_controller.ex
* creating hello_phoenix/lib/hello_phoenix_web/templates/layout/app.html.eex
* creating hello_phoenix/lib/hello_phoenix_web/templates/page/index.html.eex
* creating hello_phoenix/lib/hello_phoenix_web/views/layout_view.ex
* creating hello_phoenix/lib/hello_phoenix_web/views/page_view.ex
* creating hello_phoenix/test/hello_phoenix_web/controllers/page_controller_test.exs
* creating hello_phoenix/test/hello_phoenix_web/views/layout_view_test.exs
* creating hello_phoenix/test/hello_phoenix_web/views/page_view_test.exs
* creating hello_phoenix/.gitignore
* creating hello_phoenix/assets/brunch-config.js
* creating hello_phoenix/assets/css/app.css
* creating hello_phoenix/assets/css/phoenix.css
* creating hello_phoenix/assets/js/app.js
* creating hello_phoenix/assets/js/socket.js
* creating hello_phoenix/assets/package.json
* creating hello_phoenix/assets/static/robots.txt
* creating hello_phoenix/assets/static/images/phoenix.png
* creating hello_phoenix/assets/static/favicon.ico

Fetch and install dependencies? [Yn]
* running mix deps.get
* running cd assets && npm install && node node_modules/brunch/bin/brunch build
* running mix deps.compile

We are all set! Go into your application by running:

    $ cd hello_phoenix

Then configure your database in config/dev.exs and run:

    $ mix ecto.create

Start your Phoenix app with:

    $ mix phx.server

You can also run your app inside IEx (Interactive Elixir) as:

    $ iex -S mix phx.server

Ok, so it's telling us what to do.

cd hello_phoenix - we need to get into the newly created directory.

mix ecto.create - will create a database as per the configuration inside config/dev.exs

mix phx.server is the command we need to run to get our Phoenix server instance up and running.

However, we don't have an instance of Postgres available just yet.

Let's not dwell on this. Let's follow the instructions:

cd hello_phoenix

Next, to get a Postgres instance up and running we will need a docker-compose.yml file:

touch {docker-compose.yml,Makefile}

Now we have all the files we will need to get up and running, and fast.

Open up the docker-compose.yml file, adding in the following contents:

version: '3'

services:
  web:
    image: docker.io/codereviewvideos/docker-phoenix
    command: mix phx.server
    environment:
      - MIX_ENV=dev
      - PORT=4000
      - DATABASE_URL=ecto://postgres:postgres@postgres/dev_db
    volumes:
      - .:/app
    ports:
      - "4000:4000"
    links:
      - postgres

  postgres:
    image: postgres:9.6.5
    ports:
      - "5432:5432"

There's a lot to cover here, even though there's not that much to see.

We will be using docker-compose.yml's version: '3' syntax. You may have seen a docker-compose.yml file before, or you may not have. Generally it's fairly intuitive, I feel.

The version: 3 line is telling us that we will be using the third iteration of Docker's compose syntax. It's very similar to Version 2 syntax, with the exception of volumes_from. Again, don't worry too much about any of this, it's generally just trivia at this point.

Next, under the services key we define the things we need to make our system work.

In this case we have two services, web and postgres.

The names of these services can be changed: phoenix, and db for example. Whatever name you use here will be the name we need when trying to connect to that service, which we will get too shortly.

Next we see our web service uses the image of docker.io/codereviewvideos/docker-phoenix.

This is the exact image we created in the previous video.

Here's an alternative:

version: '3'

services:
  web:
    build: .
    command: mix phx.server
    environment:
      - ... etc

We could have put the Dockerfile for our Phoenix server in this project. The downside to doing so is that every time we run a docker-compose up, we would end up doing a build - rather needlessly.

Now, this is my personal approach. Yours may differ, and that's ok.

Next we specify the command for Docker to run inside our web container when it is brought up and online:

command: mix phx.server

This ties in nicely with what Phoenix's installation instructions told us to do. Well, sort of. We skipped a step:

Then configure your database in config/dev.exs and run:

    $ mix ecto.create

Start your Phoenix app with:

    $ mix phx.server

We will need to make sure a database server is up, and the expected database itself is available, or Phoenix will keep trying to connect, and keep spamming our logs with errors. More on this shortly.

We set up some environment variables:

    environment:
      - MIX_ENV=dev
      - PORT=4000
      - DATABASE_URL=ecto://postgres:postgres@postgres/dev_db

MIX_ENV is the way we set the environment variable for Mix. Mix is the Elixir build tool, and will handle compiling and testing, sorting out our project's dependencies, and more. There are three pre-defined environments:

  • :dev
  • :test
  • :prod

As we will be working in development mode for now, setting our mix environment to dev makes the most sense. This is also the default, so could be omitted, however I like being explicit. See more here for further information on Mix environments.

PORT=4000 is how we will change the port on which we will connect to our Phoenix server instance via our browser. 4000 is the default port. I make this explicit as if you end up running multiple instances of Phoenix via Docker, you will need to change the ports to avoid conflicts. Setting as an environment variable makes this nice and easy.

Lastly: DATABASE_URL=ecto://postgres:postgres@postgres/dev_db. That's a lot of postgres!

Ok, so let's break this down.

Firstly, we could define each of these pieces individually - username, password, host, and db name.

Instead, we use a shorthand URL syntax as it's easier in my opinion. And cooler :)

In case you aren't aware, Ecto is somewhat akin to Doctrine in Symfony, or Eloquent in Laravel. Though it's not a direct / like-for-like comparison.

Don't worry, even though we are setting up a database here, we won't need it. But beyond the basics we will, so it's worthwhile covering it.

How do we know the username and password will be postgres? It's in the docs on Docker Hub.

ecto://postgres:postgres@postgres/dev_db
echo://{username}:{password}@{host}/{db_name}

We know the username and password from the docs.

We know postgres will be the machine name, because that's what we set in our docker-compose.yml file.

The only "unknown" is the {db_name} / dev_db - which we will need to call mix ecto.create to ... create!

We will get to that shortly.

All of these environment variables, by default, are useless. We need to change our configuration to use them.

We don't need to do anything directly with MIX_ENV. This is used behind the scenes.

# config/dev.exs

config :experiment, ExperimentWeb.Endpoint,
  http: [port: System.get_env("PORT") || 4000],

Start off by changing line 10 to take either the given PORT environment variable, or if not set (||), fall back to port 4000.

Then we need to change lines 54 to 57, replacing with:

# Configure your database
config :experiment, Experiment.Repo,
  adapter: Ecto.Adapters.Postgres,
  url: {:system, "DATABASE_URL"},
  pool_size: 10

Why the different syntax:

{:system, "DATABASE_URL"}
# vs
System.get_env("PORT")

Because hey, it's programming and there are multiple ways to do things :) It's nice to see both. Feel free to use just one.

The final dev.exs we will be using looks like this:

use Mix.Config

# For development, we disable any cache and enable
# debugging and code reloading.
#
# The watchers configuration can be used to run external
# watchers to your application. For example, we use it
# with brunch.io to recompile .js and .css sources.
config :experiment, ExperimentWeb.Endpoint,
  http: [port: System.get_env("PORT") || 4000],
  debug_errors: true,
  code_reloader: true,
  check_origin: false,
  watchers: [node: ["node_modules/brunch/bin/brunch", "watch", "--stdin",
                    cd: Path.expand("../assets", __DIR__)]]

# ## SSL Support
#
# In order to use HTTPS in development, a self-signed
# certificate can be generated by running the following
# command from your terminal:
#
#     openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.example.com" -keyout priv/server.key -out priv/server.pem
#
# The `http:` config above can be replaced with:
#
#     https: [port: 4000, keyfile: "priv/server.key", certfile: "priv/server.pem"],
#
# If desired, both `http:` and `https:` keys can be
# configured to run both http and https servers on
# different ports.

# Watch static and templates for browser reloading.
config :experiment, ExperimentWeb.Endpoint,
  live_reload: [
    patterns: [
      ~r{priv/static/.*(js|css|png|jpeg|jpg|gif|svg)$},
      ~r{priv/gettext/.*(po)$},
      ~r{lib/experiment_web/views/.*(ex)$},
      ~r{lib/experiment_web/templates/.*(eex)$}
    ]
  ]

# Do not include metadata nor timestamps in development logs
config :logger, :console, format: "[$level] $message\n"

# Set a higher stacktrace during development. Avoid configuring such
# in production as building large stacktraces may be expensive.
config :phoenix, :stacktrace_depth, 20

# Configure your database
config :experiment, Experiment.Repo,
  adapter: Ecto.Adapters.Postgres,
  url: {:system, "DATABASE_URL"},
  pool_size: 10

Carrying on with our docker-compose.yml config, we have:

    volumes:
      - .:/app

The syntax here looks strange.

By using a volume we can pass through folders from our local PC to be run as though they were locally available inside the running container.

This is really powerful, especially in development.

Remember we had to install Phoenix locally, and in our Docker image?

Our local files will be run as though they were copied on to the Phoenix server container. This all happens without us having to do anything.

What the syntax is saying is:

Please use the current working directory (represented as a .) - as in the dir that contains our docker-compose.yml file - but inside the running web container, make it available as /app.

    volumes:
      - {my local directory}:/app

This is important as if we remember back to our Dockerfile for our Phoenix server, we set the current working directory:

FROM elixir:1.5.1
ENV DEBIAN_FRONTEND=noninteractive

# ... other stuff

# When this image is run, make /app the current working directory
WORKDIR /app

Next up, we want to make sure that we can access our server on port 4000:

    ports:
      - "4000:4000"

By default Docker won't expose any ports. We must explicitly expose (or open to the outside world, from the containers point of view) any ports we need. Think of this as a white list.

Here we say that when we access port 4000, we want that traffic to be sent to the containers port 4000. You can generally change the number before the colon without much issue, but to change the number after the colon will require changing the setup inside the container.

Another way to think about this would be if we had two instances of a web server - maybe one Apache, and one nginx - running on our dev box.

Both, by default expect traffic on port 80.

We could have both containers accept traffic on port 80 internally, but from the outside world (i.e. our dev PC) we could connect on ports 81, and 82 respectively:

version: '3'

services:
  apache:
    image: httpd
    ports:
      - "81:80"

  nginx:
    image: nginx
    ports:
      - "82:80"

Feel free to try this now. Create a different docker-compose.yml file in a different directory, and hit those servers:

docker-compose up

Creating network "aaaa_default" with the default driver
Pulling apache (httpd:latest)...
latest: Pulling from library/httpd
ad74af05f5a2: Already exists
3d839585b9c7: Pull complete
cf157792586a: Pull complete
c620105f0566: Pull complete
830b826a2e13: Pull complete
ec2eb5743536: Pull complete
eb53f3c09897: Pull complete
Digest: sha256:5b35d13089db73df620f4c198f5a4bfa56b8fe45a0364f343df9a26d874fef6c
Status: Downloaded newer image for httpd:latest
Pulling nginx (nginx:latest)...
latest: Pulling from library/nginx
94ed0c431eb5: Pull complete
9406c100a1c3: Pull complete
aa74daafd50c: Pull complete
Digest: sha256:788fa27763db6d69ad3444e8ba72f947df9e7e163bad7c1f5614f8fd27a311c3
Status: Downloaded newer image for nginx:latest
Creating aaaa_apache_1
Creating aaaa_nginx_1
Attaching to aaaa_apache_1, aaaa_nginx_1
apache_1  | AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.19.0.2. Set the 'ServerName' directive globally to suppress this message
apache_1  | AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.19.0.2. Set the 'ServerName' directive globally to suppress this message
apache_1  | [Fri Aug 04 10:27:09.288608 2017] [mpm_event:notice] [pid 1:tid 139699567531904] AH00489: Apache/2.4.27 (Unix) configured -- resuming normal operations
apache_1  | [Fri Aug 04 10:27:09.288955 2017] [core:notice] [pid 1:tid 139699567531904] AH00094: Command line: 'httpd -D FOREGROUND'

Now visit the two servers in your browser:

  • 127.0.0.1:81
  • 127.0.0.1:82
apache_1  | 172.19.0.1 - - [04/Aug/2017:10:27:12 +0000] "GET / HTTP/1.1" 200 45
apache_1  | 172.19.0.1 - - [04/Aug/2017:10:27:12 +0000] "GET /favicon.ico HTTP/1.1" 404 209
nginx_1   | 172.19.0.1 - - [04/Aug/2017:10:27:16 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36" "-"
nginx_1   | 2017/08/04 10:27:16 [error] 7#7: *1 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 172.19.0.1, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "127.0.0.1:82", referrer: "http://127.0.0.1:82/"
nginx_1   | 172.19.0.1 - - [04/Aug/2017:10:27:16 +0000] "GET /favicon.ico HTTP/1.1" 404 571 "http://127.0.0.1:82/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36" "-"

Fairly cool.

Of course that's the sort of trivial demo that gets people interested in Docker, but the reality of setting up a real Apache or nginx stack for a typical PHP app is more complex.

Right, so where were we?

    links:
      - postgres

Ahh yes, we want to link our web service to our postgres service.

In linking these two services we should find pinging / connecting using the given service name to actually work. The inclusion of postgres inside our links list also explicitly defines the dependency: Mr. Docker, we will need a postgres instance up and running, so please ensure it's available.

That covers the web portion. In the very next video we will get Postgres (the database) sorted out, and finally get our Dockerised Elixir and Phoenix environment up and running.

Episodes