Deploying With Ansistrano

I have a major problem. I keep waaay too many open tabs on my phone. Every time I see something interesting – via Twitter, Hacker News, or Reddit – I open the link in a tab and promise myself I will check it out more thoroughly soon.

I currently have 47 open tabs 🙁

This is only amplified by the fact I do the same on desktop. Right now, across three desktops I have over a hundred open tabs.

It’s becoming an epidemic.

Anyway, once in a while, one of these tabs becomes useful.

Recently I hit on a problem whereby I needed to deploy a bunch of individual JavaScript files – node.js scripts – to a server to be used as Rabbit MQ workers.

I had a bunch of requirements for these workers:

  • start with specific flags ( --harmony-async-await )
  • restart automatically if the server reboots
  • restart if the script crashes
  • can run multiple instances

And so on.

These turned out to be the superficial problems – and I say this because there’s a tool out there that already nails this problem – PM2.

Initially I thought these would be the hard problems.

What I hadn’t banked on was how much of a royal pain in the backside it might be to deploy my node.js scripts to dev / prod / wherever.

 

My requirements are fairly straightforward – they could be solved by using rsync. However,  rsync becomes unmanageable as a project grows.

There’s the issue of remembering the right command, and then duplicating the command – altering slightly – for the prod deploy.

And what if it goes wrong? Well, you have to handle that yourself.

Call me spoiled, but having become accustomed to Deployer (Matt did a fantastic job on this course btw, you should check it out), I now use that as my baseline for deployments.

I have a similar tool I use on JavaScript projects called Flightplan. It offers a decent level of functionality, but with one major issue (from my p.o.v):

It is a pain to deploy more than one directory.

Flightplan works on the assumption – as best I can tell – that you will be running your project through some webpack-style setup first, producing a dist directory which contains everything you need to boot your single page app, or whatever.

This is cool, but I needed to run many different worker scripts – all ideally from one directory.

As best I understand it, webpack allows this via it’s multiple entry options, but I’m not using webpack. Actually, I tried to use webpack but it threw out a bunch of errors right away and I gave up.

I also tried Deployer. But that didn’t work much good either. JS mixed with PHP leads to mess.

Enter Ansistrano

Ok, so all that was a very long-winded precursor to the eventual solution.

However, I felt I needed to do justice to how much I have struggled to get this thing working. It’s taken 5 hours… ouch.

Needless to say I tried to give up on getting Ansistrano at numerous times (see how I ended up at Deployer, Flightplan, webpack etc).

In the end though, I cracked it. So here goes:

Firstly, my playbook:

Pay special attention to the  ansistrano_git_repo entry, whereby I needed to add the prefix of  ssh:// to make this work. If you don’t, you will find Ansible doesn’t understand the path you are providing, and blanks it out instead :/

I guess I wasn’t the only person to notice this.

Also, note that the typical git path given by gitlab will contains colons, which need to be replaced with spaces:

 

Now note, this is an Ansible issue, not an Ansistrano issue.

This should be enough to get most of the way there.

However, I hit upon another issue.

No matter what I did, all the Ansistrano managed folders were being created as root .

Since the days of yore, I have been using the same set of flags on my runs of  ansible-playbook, and today I was well and truly bitten on the backside:

Ultimately this command sees me through. I’ve started using -vvvv on every playbook run as it saves me having to re-run when things inevitably go wrong. Also, for the love of God, use snapshots before running.

But yeah, my issue was I was running with the additional flag of  -s which forced the playbook to run as root. Silly me.

Anyway, early signs are promising. It all works. I just wish it hadn’t taken me so much time to figure out these problems. Hopefully though, by sharing I can save someone some hassle in future.

Upgrading to Ansible 2.1 on OSX

I have Ansible installed on a variety of machines – primarily Ubuntu, but also for setting up the virtual machines / servers I use when recording, I also have Ansible installed on OSX.

Up until today I was running Ansible v1.9.1, but then I hit on an issue with my nginx sites not creating properly (for some reason which I did not track down), and decided the best course of action would be to upgrade all the submodules I use as the basis of my playbook infrastructure.

Just to explain this a little further – when I first started using Ansible I would  git clone any interesting repository to my local machine, and then take ownership of that repo into my own project. There are pros and cons to this approach:

Pros

  • I could make changes to the repos to suit my needs
  • I knew the repo contents wouldn’t change unless I changed them
  • Everything was very explicit and obvious to new people

Cons

  • I created a lot of work for myself

Yeah, ‘busy’ work such as managing an ever expanding list of other people’s projects was not a fun thing. Instead, I switched to ‘importing’ other peoples projects by way of git’s submodules, which works a treat, but there’s a drawback:

Other people have a habit of changing things.

My mistake was to blindly update all my submodules to their latest versions:

This worked fine, but it turned out that some of my tracked submodules now required Ansible 2 to run.

No problem, I thought, let’s just upgrade to Ansible 2.

How naive.

The first mistake I made in this process was to try and update an OSX installation of Ansible by way of using  pip install ansible --upgrade.

Yeah… not so much. Plenty of errors here:

Which is honestly fair enough – this isn’t how I’d installed Ansible originally (my poor grep skills fooled me), and this led me down some dark alleys of trying to manually fudge the upgrade of dependencies I thought were at fault – pycrypto, paramiko, and others.

The command that worked was simple:

Ok, so cool, up to date with Ansible, but the playbook still wouldn’t run properly.

Quite frustrating. I knew I could SSH onto the box, so it was likely not my server at fault.

A simpler Ansible ping command helped a little:

From here I found a helpful github issue which led me to this:

Note the inclusion of:

But this needed a little further help to start working:

After that, the ping and playbook command would run again.

Jackpot.

How I Fixed: File is absent cannot continue

Lately I’ve been tidying up my Ansible playbook scripts ahead of a forthcoming project deployment. Lots of change, including upgrading to PHP7 across the board.

Pretty cool, and exciting, but quite a lot of work all the same.

I decided to take the opportunity to restructure my log files. I mean, that’s a good way to spend a Friday night, right? Of course it is.

The relevant part of the nginx playbook is:

It’s admittedly been a while since I’ve been regularly hands-on with my Ansible setup, so I was a little rusty. See if you can spot the error in the above.

The with_items section lives in my  host_vars/my-symfony-3.dev file, and looks like:

So, anyway the error that was being spat out was as follows:

Hopefully it make it easier to Google for this for someone in the future.

Anyway, the solution will make you kick yourself. I can’t imagine that many people will make this typo, but you never know:

Yeah… d’oh. RTFM.

Though to be fair, I had RTFM, it was just a long time ago, and I guess I rely too heavily on IDE code completion. When it comes to Sublime + YAML, I am going to make mistakes 🙂

If this looks interesting to you, and you want to know more about Ansible, be sure to check out the tutorial series I did here at Code Review Videos.

It’s shameless self promotion, I know, but hey, it is free. And hopefully you find it useful.

Installing PHP7 on Ubuntu with Ansible

In a bid to keep updated with the latest and greatest in PHP-land, I have been slowly updating my Ansible set-up to accommodate PHP7.

Fortunately, pretty much all of the hard work has been taken care of by Jeff Geerling, with his fantastic geerlingguy/php role. A mighty big thank you is in order.

But I’m writing this as I struggled with a particular issue getting my newly updated playbook to work properly.

The issue I had was that I had the following:

And whilst everything looked good, whenever I ran the playbook:

The cheeky little blighter would not run the task to add the repository before it would try and run the  roles content. Needless to say, red error text appeared in quantity.

Annoyingly, I could comment out the roles section, and the task would run – but as soon as I put the role definition back in, that would always run first.

I must admit, this was a new one on me.

However, the solution is incredibly simple:

Please note: this config is still very much a work in progress for me. I found this particular set of config from this ticket on Jeff Geerling’s Ansible Role PHP repo.

Yes, there are  pre_tasks and  post_tasks available – new ones on me, but right there in the docs. You may need to ctrl+f to find the exact examples, as I can’t directly link to the specific section unfortunately.

Very useful anyway, and with that, my task started behaving exactly as I wanted.

Ansible is absolutely awesome. If you haven’t already done so, be sure to check out my Ansible tutorial course here at Code Review Videos. It’s completely free.