I want to start by saying thank you to everyone who got in touch regarding last week’s post on learning how to deploy with Docker. Judging from the feedback I received, I’m not the only one who has found deploying docker to be difficult.
Last week I mentioned how I was 11 days into my docker deployment exercise, and at that point the end did not seem to be in sight. Thankfully this week, although now at 17 days (at the time of writing), I can say “GREAT SUCCESS!”
The truth is, it has been an absolute pain to get this far. The short version of events is that I wanted to deploy a fairly standard Symfony 3 stack:
- Symfony 3 JSON API
- MySQL database
- nginx web server
- Lets Encrypt for lovely free SSL
- RabbitMQ for queuing
- Node w/ PM2 for queue workers
And I hit upon a fair few problems doing this with Ansible.
Now, don’t get me wrong: Ansible is fantastic.
There is – however – a major stumbling block with Ansible. If you make a mistake, reverting that mistake can be quite painful. My solution to this has often been destroy the server, and simply re-run the Ansible playbook(s) again until a working stack is back up and running.
But here’s the thing: This is terrible in production.
“No kidding, Chris” – everyone, everywhere.
I get this. It’s not a solution. The alternative is to jump onto the box, and start hacking at the config until you get a working system again. Only now you have a problem – you need to ensure your Ansible playbook / role setup will reliably reproduce this exact same setup. And that can be really, really tricky.
Pretty much the only way to guarantee you have nailed this is to, ahem, wipe the box and re-run the playbook.
See, in development you have a nice easy life, for the most part. If things go wrong, you can flatten everything, and start again. It costs you time, and time (as they say) is money. But overall, the cost is cheap.
As part of your development process you can add in extra pieces to your stack without much concern. Need RabbitMQ? There’s a role for that. Need Redis? There’s a role for that. Need a firewall? Sure, there’s a role for that – but do you really want to bother with a firewall in development?
And as a firewall in development seems a little overkill, I am often sorely tempted to skip this step. Hey, I have a lot on my plate and if I don’t need it right now, then I put it off.
In a similar vein, SSL in development is a total luxury. Without a public DNS record I can’t do any LetsEncrypt magic. I could do a self signing effort (there’s very likely an Ansible role for that, too), but what’s the point? If I use a self signed cert in dev, but LetsEncrypt in prod, now I have two different environments. Actually, using LetsEncrypt in prod, and not using it in dev is still two different environments, but somehow that feels … better?
Ultimately I have always ended up with two varying environments. Dev is a stripped down, and subtly different version of prod.
Anyway, back to my deployment woes. The straw that broke the camel’s back for me was in deploying the Node queue workers. The queue workers are fairly simple Node JS scripts. The gist of it is that they are long running processes that get given messages (think: JSON data) from RabbitMQ. They parse the JSON, and take some appropriate action.
As ever with software, things inevitably go wrong. When things go wrong with Node, it has the tendency to throw it’s hands up in the air and give up. My scripts behave like a pop star with a sore throat. PM2 is a process manager designed to sit and watch these processes on a 24/7 basis, and pro-actively restart any that die, for any reason. To continue the shaky analogy, PM2 is the venue manager, with two burly thugs waiting to duff up said pop star if the awaiting concert fans aren’t promptly entertained.
Before PM2 could do its thing, I needed a way to reliably get it installed on my servers. You guessed it – there’s an Ansible role for that.
Each role adds (yet another) layer of abstraction around whatever particular task you are trying to achieve. This layer means an extra bunch of potential steps that may or may not be causing any problems you might encounter. In my case, my problem was that whilst my code would deploy, and PM2 would manage it, it was always an out-of-date version.
To try and fix this, I started using Ansistrano (hey, another Ansible role!) to manage the deployment of my code, which triggered PM2 to hopefully reload my latest code changes… except it didn’t.
Somewhere along the way, I’d become many layers deep into a problem that I’d used all these tools to try and avoid.
In hindsight, I reckon if I’d stuck with the problem for the past two and a bit weeks, I would likely have found a solution. But the truth is, my confidence in my deployment process was at a low. With no traffic to the site, it wasn’t hurting any end users (thank the Lord!), but I could quite easily picture a future point where I had either deployed and made a mess, or worse, become so scared of deployment that I had essentially stopped new development to avoid the issue.
I know I’m not alone in this. In fact, I’ve worked with numerous organisations of many sizes that suffer from this very problem.
Staring me in the face, right there on the PM2 web site, was the section on “Docker Integration”.
Again, in hindsight, it probably would have made more sense to start small and Dockerise just the Node Script / PM2 deployment.
But that’s not what I did. No siree. Instead, I decided – to heck with this – let’s go the whole hog and move this entire stack to Docker.
So yeah, it’s been a long, hard two (and a bit) weeks.
But I’m there. Well, 95% of the way there. See, very late yesterday evening it suddenly dawned on me that I’d neglected to include the WordPress blog which I’m using on the site root. Symfony is brilliant, don’t get me wrong, but use the right tool for the job. WordPress might get a lot of stick for its code, but the product that the end user sees is brilliant.
Putting WordPress into Docker should be easy right? There’s tons of tutorials on this. Heck, even the official Docker Compose tutorial has a guide on how to do just this.
Actually though, it’s not easy. Sure, it’s different to using Ansible. Docker gets you up and running faster, there is no denying that. It also uses a whole bunch less memory and disk space than having to provision a bunch of virtual machines. But really, it’s just a different set of difficult problems.
Taking WordPress as the example: where do you store user uploads? What about new plugins?
These are things you don’t give much of a second thought to in a ‘traditional’ deployment. In Docker, these are headscratchers. You’re discouraged from using Volumes in production, but without them you can’t persist any data…
Most every tutorial I have found completely gloss over these trivial details :/
Anyway, after all this I do have some working solutions to these problems which I am going to share with you. From the plan I’ve made for this series it could become sprawling. I don’t want this to be the case. I’d rather keep it concise, but the truth is there is an absolute ton of stuff that you need to know to actually work with Docker.
This series is going to have a particular focus on deploying Symfony with Docker. We will also cover Rancher, a tool for managing your containers in production. In my opinion, after deploying Symfony with Docker you will have encountered a whole bunch of real world problems that make working with Docker on pretty much any other code base a lot easier.
I’d still love to hear from you if you are working with Docker in any way – whether just tinkering with it; working with it only in development; or have fully embraced Docker and put it into production. I have plenty more war stories to share, and whilst I can’t promise to have answers to your questions, if I can help in any way, I will do my best to try.
Video Update
This week there have been three new videos added to the site:
Towards the end of recording the Symfony Workflow Component Tutorial series I spotted an upcoming feature in Symfony 3.3, which is Workflow Guard Expressions.
As a quick recap, a Guard is a way of blocking a Transition. A transition is the process that objects going through your workflow will need to pass through to get from place to place.
Earlier in the series we covered creating separate Symfony Services to ‘contain’ your Guard logic.
With Guard Expressions you may be able to replace these standalone services with simple, human readable one-liners. It won’t always be the solution, but I’ve found them incredibly useful for two reasons:
* They cut down on code (less code, less bugs)
* They put your guards right inside your workflow definitions (easier to understand at a glance)
They are cool, and useful, and if you’re using the Symfony Workflow Component then I encourage you to check them out.
and
In this two part series we cover a frequently requested concept – working with CSV data inside a Symfony application.
This series was created in response to a question from a site member regarding how to import CSV data, and turn it into related Doctrine entities.
When thought about as one process – importing and converting – it can be quite overwhelming. But if you break this process down into two tasks:
- reading a CSV file;
- converting each row into related Doctrine entities
Then it becomes a lot easier.
The approach given in these videos is a not intended for production use. You will need to expand on these concepts to fit your own needs. The aim here is to cover the high level, rather than “this is the implementation you should be using!”, which I disagree with. Every project is unique and has its own requirements. Use this as a possible source of guidance.
All of this weeks videos are free, and have been put up on YouTube also.
Thanks for reading, and have a great weekend.
Until next week, happy coding!
Chris
