Doctrine Cache Money

In this video we are going to take a look at how we can leverage Doctrine Query Cache to increase our applications performance.

We do this by way of reducing the number of repetitive queries that may run and re-run, when the data changes so infrequently that we could have just re-used the result from the first run on the successive calls.

A good example of this would be the Code Review Videos Course List, and also the Video List.

Both of these examples change infrequently, compared to the number of times per day they are accessed.

Taking the Video List, I will generally add a new Video once per day. However, every time the Video List is accessed, the same query is run - and in pseudo SQL, this would be something akin to:

SELECT * FROM videos_table ORDER BY publication_date DESC

Now, of course, that is not the real query, and I use Doctrine, so it's not like I'm writing raw SQL either, but as an example, that's close enough.

Per day, the Video List page may be accessed between 70-100 times. I'd like it to be more :)

The thing is, without some caching logic, that would be 70-100 runs of the same query, over and over, returning the same data.

Granted, each query and result set is tiny compared to some vast industrial mega application, but it's still pointless overhead. As the old programmers maxim would say: "Don't Repeat Yourself". In this case, we want our pal the MySQL server to save itself a bit of work, and in turn, do our bit for the environment by not wasting CPU cycles and disk spinny upy, disk spinny downy.

As such, we can enable caching.

Caching is a fancy way of saving data so it can be re-used later.

There are a number of caching drivers included with Doctrine/Common to match up with the more common caching solutions found in PHP.

The drivers we are interested in are the array and apc drivers, but memcache, memcached, xcache and service options are available. The service option allows us to create our own Symfony Service for caching, which you can base off the CacheProvider class, should you have a very specific need.

The Three Doctrine Caches

By default, none of our queries are cached. And none of our query results are cached. And our entity metadata is re-read, on every query we run.

What's probably most confusing about that is that there are three different things that can be cached - not just the result, which is what I initially focused on.

See, caching the result seems obvious. We do a query, we care about the result. So we save the result and next time we run that query we get the saved result instead of re-running the query again.

Makes sense.

But what about the stage where we transform our DQL into raw SQL? Do we need to do that every time we run a query? Likely not.

So we can cache it. And interestingly, this is referred to as the query cache.

The result cache is just that, the cached result of the actual query we ran. That way we aren't querying the database, and we aren't having to hydrate a bunch of objects again.

It's pretty easy to think about query cache and result cache as one and the same. But they aren't. And it's the sort of thing a pedant would pick you up on during an important client meeting. But that won't catch you out any more.

And then there is metadata cache. That's all the mapping information we put into our entities, along with various other bits that tell Doctrine's ORM how to use our entities behind the scenes.

The likelihood of our metadata changing frequently is pretty slim, so again, having to constantly read and re-read this data is needless.

But do be careful, as this can catch you out later.

Enabling Doctrine Cache

Now we know about the three types of Doctrine Cache, we can make a start on using them.

Enabling caching in Doctrine when using Symfony is really quite simple.

Firstly, you need to make sure you have whatever underlying caching implementation you are using (apc in our case) installed and working. APC may not be the best for you in production, so be sure to choose an option that better suits - Redis is great for bigger projects, whereas Memcached will likely suffice on smaller projects. Be sure to research both, and other, options rather than relying solely on the advice of a tutorial aimed at being as generic and accessible as possible.

Installing APC should be pretty simple on Ubuntu:

sudo apt-get install php-apc

Then restart your web server.

If struggling, check out this tutorial.

With APC installed, we should be good to go.

Next up, we need to add in the relevant config into our config.yml file:

# config.yml
doctrine:
    orm:
        # *snip snip*
        metadata_cache_driver: apc
        query_cache_driver: apc
        result_cache_driver: apc

We can leverage the way Symfony reads our configuration files to override these settings depending on our environment.

As such, we can change the config for app_dev.php by overriding the values inside config_dev.yml:

# config_dev.yml
doctrine:
    orm:
        metadata_cache_driver: array
        query_cache_driver: array
        result_cache_driver: array

This will ensure we use APC in production, but Arrays inside our dev environment.

Be sure to clear your Symfony caches after doing this:

php app/console cache:clear
php app/console cache:clear --env=prod

Now that we have caching enabled, we can move on to actually using it.

Using Doctrine Cache for Caching Our Results

By default, just because we have enabled our Doctrine Cache, it won't necessarily be used.

The convenience methods such as find, findBy, and findAll, won't be cached.

Instead, we must switch to using the Query Builder, which not only gives us fine grained access to our query, but allows us to insert the method which tells Doctrine to cache our data.

To enable our query's result to be cached, we need to add in useResultCache(true) to our query builder call.

$topics = $qb->select('t', 'r')
  ->from('AppBundle:Topic', 't')
  ->join('t.replies', 'r')
  ->getQuery()
  ->useResultCache(true)
  ->getResult()
;

We can also tell Doctrine exactly how long to cache our results for by adding in a second parameter to our call to useResultCache(). For example, to cache our result for a minute, we would do the following:

$topics = $qb->select('t', 'r')
  ->from('AppBundle:Topic', 't')
  ->join('t.replies', 'r')
  ->getQuery()
  ->useResultCache(true, 60)
  ->getResult()
;

Pretty easy.

Array Cache Gotcha

As you will see in the video (around the 4:45 mark), when we use the array cache driver, the caching only lasts for the single request we perform.

This will show up predominantly in the way that refreshing the page always seems to re-run the query.

This is great for development where we likely will be making many changes, but not so great for production. So don't bother with using the array cache drive in production.

And just be aware of the side effects of using array caching wherever you use it.

Interesting info: array is the default driver used if we don't specify any other.

Clearing Doctrine Cache

For numerous reasons, clearing your Doctrine Caches can be important.

Perhaps the biggest would be when you make a change to an entity, but the changes that are working quite happily in the development environment don't seem to update or reflect in production.

The likely cause of this is our Doctrine Caching layer.

We can force Doctrine to erase its cache by using one of the three following commands:

doctrine:cache:clear-metadata         #Clears all metadata cache for an entity manager
doctrine:cache:clear-query            #Clears all query cache for an entity manager
doctrine:cache:clear-result           #Clears result cache for an entity manager

Further Reading

Of course there's tons more to be learned about Caching.

The following links may be useful to you:

• http://symfony.com/doc/current/reference/configuration/doctrine.html
• http://doctrine-orm.readthedocs.org/en/latest/reference/caching.html
• http://doctrine-orm.readthedocs.org/en/latest/reference/second-level-cache.html