Fixing The Fixtures


Getting to a tested Wallpaper Upload implementation has been relatively tough going. Now that we are there, we still have a few more tasks to complete before we can consider this part of the project "done". Of course, even when it's "done", it won't really be truly done. Such is the life of a software developer :)

We now need to tackle two further problems:

  • How to edit existing Wallpapers
  • How to delete existing Wallpapers, and ensure the associated wallpaper image file is removed as part of the process

Much like before, we are first going to prototype our approach, and then switch back to our tested approach to see how things differ.

The idea behind this process is to have one section of our site that we can compare and contrast, weighing up the pros and cons of both approaches. This will come a little later. For now, let's keep implementing.

'Fixing' The Fixtures

By implementing the 'Create' form, we've done most of the hard work for the 'Update' screen already.

Unlike in a typical Symfony app, we don't need to worry about querying for the Wallpaper in our database and then passing in the resulting entity into our form. EasyAdminBundle will take care of this for us.

What we do need to concern ourselves with is:

  • Can we edit?
  • On the edit form, how can we see the existing image that's associated with this Wallpaper entity?
  • Does our data save correctly?

Most of this seems like it should behave without too much fuss. The hard part seems to be in re-displaying our Wallpaper image file when going to the 'Update' form.

Let's start off by switching branches back to where we were before doing any of the PhpSpec implementation:

git checkout vid-20 -b untested-wallpaper-edit

Switched to branch 'untested-wallpaper-edit'

This command does two things.

Firstly, it switches our codebase back to as it was at the point where we tagged vid-20. This was the end of our prototype phase for Wallpaper Upload.

Secondly, it creates us a new branch called untested-wallpaper-edit. Feel free to name the branch anything you like, so long as it is unique in your project.

That's our code sorted out, but we still have a bigger issue to tackle: the database.

During the last 6 videos we have made a variety of changes to our Wallpaper entity, and the way in which it expects to work with files. This has meant changes to the database schema which are incompatible with our untested approach.

Fortunately, we created some fixtures quite early on in the project. But unfortunately, these fixtures won't work without a bit of a tweak.

Let's do as much as we can, and then address the problems as they arise:

php bin/console doctrine:database:drop --force

Dropped database for connection named `wallpaper_site`

php bin/console doctrine:database:create

Created database `wallpaper_site` for connection named default

php bin/console doctrine:migrations:migrate

# * snip *

  ++ finished in 0.15s
  ++ 4 migrations executed
  ++ 6 sql queries

So far, so good. We have dropped and recreated our database, and migrated from nothing to our expected database schema state.

Next, we would likely be quite pleased with ourselves if the fixtures would load for us, correctly repopulating our shiny fresh database with some known data:

php bin/console doctrine:fixtures:load

Careful, database will be purged. Do you want to continue y/N ?y
  > purging database
  > loading [100] AppBundle\DataFixtures\ORM\LoadCategoryData
  > loading [200] AppBundle\DataFixtures\ORM\LoadWallpaperData

  [Doctrine\DBAL\Exception\NotNullConstraintViolationException]
  An exception occurred while executing 'INSERT INTO wallpaper (file, filena
  me, slug, width, height, category_id) VALUES (?, ?, ?, ?, ?, ?)' with para
  ms [null, "abstract-background-pink.jpg", "abstract-background-pink", 1920
  , 1080, 1]:
  SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'file' cannot
   be null

  [Doctrine\DBAL\Driver\PDOException]
  SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'file' cannot
   be null

  [PDOException]
  SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'file' cannot
   be null

Oh my.

This is a touch confusing. We haven't changed the fixtures files since we created them, and they worked before. So what gives?

Well, in between we have added in the WallpaperUploadListener. And as we have covered, that listener will be triggered before every entity persistence operation (prePersist).

What's happening here is that our listener is being triggered, even though we aren't doing a typical Wallpaper Upload as we know it.

What this means is we likely need to rethink the name of our WallpaperUploadListener. Maybe we should drop the Upload part altogether.

This also means we need to update our fixtures. I don't think there's a way round this. Unfortunately this is a bit of fiddly, nasty process.

Let's step through it.

<?php

// /src/AppBundle/DataFixtures/ORM/LoadWallpaperData.php

namespace AppBundle\DataFixtures\ORM;

use AppBundle\Entity\Wallpaper;
use Doctrine\Common\DataFixtures\AbstractFixture;
use Doctrine\Common\DataFixtures\OrderedFixtureInterface;
use Doctrine\Common\Persistence\ObjectManager;
use Symfony\Component\HttpFoundation\File\UploadedFile;

class LoadWallpaperData extends AbstractFixture implements OrderedFixtureInterface
{
    public function load(ObjectManager $manager)
    {
        $wallpaper = (new Wallpaper())
            ->setFile(
                new UploadedFile(
                    __DIR__ . '/../../../../web/images/abstract-background-pink.jpg',
                    'abstract-background-pink.jpg'
                )
            )
            ->setFilename('abstract-background-pink.jpg')
            ->setSlug('abstract-background-pink')
            ->setWidth(1920)
            ->setHeight(1080)
            ->setCategory(
                $this->getReference('category.abstract')
            )
        ;

        $manager->persist($wallpaper);

        // more fixture data here

For each of our wallpaper entities, we're going to need to add in the call to setFile.

Each setFile call needs be to given an instance of Symfony's UploadedFile, correctly populated with two bits of data:

  • The path to the image file
  • The 'real name' of the image file

Fortunately we have all this information to hand.

This should be good, right?

We know our image files exist in the web/images directory, and that big long messy path:

__DIR__ . '/../../../../web/images/abstract-background-pink.jpg',

is going to look at the current directory (__DIR__), and concatenate whatever that resolves too, then go up four directories (/../../../../), back to our site root, and down into the web/image directory to where our file lives.

If you're thinking this is awful, then I'm right alongside you :)

Before going too much further, let's try this out:

php bin/console doctrine:fixtures:load

Careful, database will be purged. Do you want to continue y/N ?y
  > purging database
  > loading [100] AppBundle\DataFixtures\ORM\LoadCategoryData
  > loading [200] AppBundle\DataFixtures\ORM\LoadWallpaperData

  [Symfony\Component\Filesystem\Exception\IOException]
  Cannot rename because the target "/path/to/my/wallpaper/app/
  ../web/images/abstract-background-pink.jpg" already exists.

Right, yeah, that doesn't work.

Our WallpaperUploadListener is trying to move the given file on to itself. Our real working system will - hopefully - never encounter such weird situations. But, during dev, these kinds of things often happen. At least, they do to me.

This is a tricky situation. At least, it as best I am aware.

To fix this is going to involve a hack. You could look to alternative solutions - maybe a bash script or similar - but we're working in PHP, and so I'm going to attempt to 'fix' this using PHP. Hold on to your hats, it's hack time.

Firstly, we need to copy the current web/images contents to some location we control. These images are going to be our 'fixture' images. When we recreate the project from fixtures, these known images should be used to populate the web/images directory.

cd /path/to/my/project/root
mv ./web/images/ ./src/AppBundle/DataFixtures

All our image files should now live in the /src/AppBundle/DataFixtures/images directory.

If we go ahead and update our UploadedFile entry to use this new location, we will hit upon another problem.

When our WallpaperUploadListener::prePersist method is triggered (as discussed above), our image file will be moved from wherever it is currently, to the new location.

This presents a major problem to us.

As this moves the file, the file is... ahem, moved.

The next time we run the fixtures, guess what? Right, the file is no longer in the /src/AppBundle/DataFixtures/images directory. Yikes.

Ok, so we need a mechanism to cope with this issue also.

<?php

// /src/AppBundle/DataFixtures/ORM/LoadWallpaperData.php

namespace AppBundle\DataFixtures\ORM;

use AppBundle\Entity\Wallpaper;
use Doctrine\Common\DataFixtures\AbstractFixture;
use Doctrine\Common\DataFixtures\OrderedFixtureInterface;
use Doctrine\Common\Persistence\ObjectManager;
use Symfony\Component\HttpFoundation\File\UploadedFile;

class LoadWallpaperData extends AbstractFixture implements OrderedFixtureInterface
{
    public function load(ObjectManager $manager)
    {
        // -------------- setup -----------------------------------
        $imagesPath = __DIR__ . '/../images';
        $temporaryImagesPath = sys_get_temp_dir() . '/images';
        echo 'Copying images to temporary location' . PHP_EOL;
        exec('cp -R ' . $imagesPath . ' ' . $temporaryImagesPath);
        // --------------------------------------------------------

        $wallpaper = (new Wallpaper())
            ->setFile(
                new UploadedFile(
                    $temporaryImagesPath . '/abstract-background-pink.jpg',
                    'abstract-background-pink.jpg')
            )
            ->setFilename('abstract-background-pink.jpg')
            ->setSlug('abstract-background-pink')
            ->setWidth(1920)
            ->setHeight(1080)
            ->setCategory(
                $this->getReference('category.abstract')
            )
        ;

        $manager->persist($wallpaper);

        // more fixture data here

        // --------------- tear down ---------------------------
        echo 'Removed images from temporary location' . PHP_EOL;
        exec('rm -rf ' . $temporaryImagesPath);
        // -----------------------------------------------------

The idea I'm going for here is that whenever we run our fixtures, we will start by copying all the images inside our /src/AppBundle/DataFixtures/images to some temporary location.

This temporary location should be OS independent thanks to sys_get_temp_dir.

We then use the temporary file path as our image path, so that when it is moved we don't actually move our original image.

The rest of the fixtures behave as normal.

Finally at the end we do a little clear up.

This obviously has a bunch of negatives / drawbacks. I'm open to a better implementation here.

The biggest problem with this that I can see, aside from using exec ... and it being horrible, is that the next time we want to run our fixtures, we would need to delete the web/images directory by hand. We could add this process in to this script but this is already feeling bad, so I'm not going to do that.

Another downside is that this process of updating the Wallpaper fixtures needs to be done for each and every Wallpaper we are creating. Boo.

We can make this approach a little more palatable by using Symfony's Filesystem component, as covered towards the end of the video. In my opinion using the filesystem is simply adding a layer of 'paint' over the raw process beneath.

Anyway, at this point our fixtures should be loading again:

php bin/console doctrine:fixtures:load

Careful, database will be purged. Do you want to continue y/N ?y
  > purging database
  > loading [100] AppBundle\DataFixtures\ORM\LoadCategoryData
  > loading [200] AppBundle\DataFixtures\ORM\LoadWallpaperData

Remember, if you need to reload these fixtures in the future you will need to delete the contents of your web/images directory.

The only justification I have for doing this is that these are fixtures. They are designed to set up the system in a known way. They will not ever be run in production, nor hopefully, very often.

To ensure that these fixtures can never be run in prod, we could make a change to our composer.json file:

    "require": {
        "php": ">=5.5.9",
        "doctrine/doctrine-bundle": "^1.6",
        "doctrine/doctrine-cache-bundle": "^1.2",
        "doctrine/orm": "^2.5",
        "incenteev/composer-parameter-handler": "^2.0",
        "javiereguiluz/easyadmin-bundle": "^1.16",
        "knplabs/knp-paginator-bundle": "^2.5",
        "sensio/distribution-bundle": "^5.0",
        "sensio/framework-extra-bundle": "^3.0.2",
        "symfony/monolog-bundle": "^3.0.2",
        "symfony/polyfill-apcu": "^1.0",
        "symfony/swiftmailer-bundle": "^2.3.10",
        "symfony/symfony": "3.2.*",
        "twig/twig": "^1.0||^2.0"
    },
    "require-dev": {
        "phpspec/phpspec": "^3.4",
        "sensio/generator-bundle": "^3.0",
        "symfony/phpunit-bridge": "^3.0",
        "doctrine/doctrine-fixtures-bundle": "^2.3",
        "doctrine/doctrine-migrations-bundle": "^1.0"
    },

The change here is to move the fixtures and migrations bundles into the require-dev section. I should have done this when adding the bundles by using e.g.:

composer require --dev doctrine/doctrine-fixtures-bundle

Ok, enough with the warning. If you don't like this approach, by all means use a different one. If you know of a better approach to this, do please share.

Code For This Course

Get the code for this course.

Code For This Video

Get the code for this video.

Episodes

# Title Duration
1 Introduction and Site Demo 02:14
2 Setup and a Basic Wallpaper Gallery 08:43
3 Pagination 08:24
4 Adding a Detail View 04:47
5 Creating a Home Page 11:14
6 Creating our Wallpaper Entity 07:50
7 Wallpaper Setup Command - Part 1 - Symfony Commands As a Service 05:57
8 Wallpaper Setup Command - Part 2 - Injection Is Easy 08:53
9 Wallpaper Setup Command - Part 3 - Doing It With Style 05:37
10 Doctrine Fixtures - Part 1 - Setup and Category Entity Creation 08:52
11 Doctrine Fixtures - Part 2 - Relating Wallpapers with Categories 05:56
12 EasyAdminBundle - Setup and Category Configuration 06:02
13 EasyAdminBundle - Wallpaper Setup and List View 07:46
14 EasyAdminBundle - Starting with Wallpaper Uploads 05:57
15 Testing with PhpSpec to Guide Our Implementation 03:39
16 Using PhpSpec to Test our FileMover 05:34
17 Symfony Dependency Testing with PhpSpec 08:47
18 Defensive Counter Measures 06:33
19 No Tests - Part 1 - Uploading Files in EasyAdminBundle 11:01
20 No Tests - Part 2 - Uploading Files in EasyAdminBundle 07:05
21 Don't Mock What You Don't Own 09:36
22 You've Got To Take The Power Back 07:36
23 Making Symfony Work For Us 08:56
24 Testing The Wallpaper File Path Helper 15:11
25 Finally, It Works! 14:56
26 Why I Prefer Not To Use Twig 16:51
27 Fixing The Fixtures 11:20
28 Untested Updates 14:30
29 Untested Updates Part Two - Now We Can Actually Update 06:33
30 Adding an Image Preview On Edit 12:31
31 Delete Should Remove The Wallpaper Image File 11:02
32 Getting Started Testing Wallpaper Updates 10:02
33 Doing The Little Before The Big 08:13
34 Tested Image Preview... Sort Of :) 07:36
35 Finishing Up With a Tested Wallpaper Update 10:41
36 Test Driven Wallpaper Delete - Part 1 11:06
37 Test Driven Wallpaper Delete - Part 2 11:57
38 EasyAdminBundle Login Form Tutorial 08:01