What will our JSON API actually do?


To begin with, we need an understanding of what our JSON API should do. It doesn't matter whether our JSON API is created in PHP, or JavaScript (nodejs), or Elixir, or Java... or any other language. The functionality from an end user perspective should be identical.

To ensure consistency with whichever platform or language we use to create our JSON API, we're going to start by defining a test.

We're going to use PHP, and specifically, Behat to define our tests.

The aim of this tutorial is not to teach you the ins-and-outs of Behat. You do not need to learn everything about Behat to use or modify this test file.

One of the absolute joys of Behat is in its use of Gherkin syntax.

Gherkin is human readable. How human readable? Well, here we go:

Feature: Provide a consistent standard JSON API endpoint

  In order to build interchangeable front ends
  As a JSON API developer
  I need to allow Create, Read, Update, and Delete functionality

  Scenario: Can get a single Album
    Given I request "/album/1" using HTTP GET
    Then the response code is 200
    And the response body contains JSON:
    """
    {
      "id": 1,
      "title": "some fake album name",
      "track_count": 12,
      "release_date": "2020-01-08T00:00:00+00:00"
    }
    """

Ok, now being completely honest, writing the business descriptions - the "In order to..., As a..., I need..." part - is often the hardest thing for me. I'm not overly concerned about what I've written here, as this isn't really a true use of BDD.

Next we have a scenario.

This is where things get interesting.

We can see we are working with Albums. This could really be anything, but gives us an opportunity to play with text, numbers, and datetimes.

Text and numbers are fairly intuitive. Date / times are a pain and will require additional work.

There's some other interesting things to note here:

We will get Albums by ID. If you wish, you can add in functionality to get albums by name, or some other property. Consider doing this as a learning exercise.

Also, our field names are snake_case. Symfony typically works with camelCase, so we'll need to address this. It's not essential, but my preference for API output is snake case.

Lastly we want to use this Behat test for any of our back end JSON API implementations. This means we will use a PHP / Behat combo to test a JavaScript / nodejs JSON API implementation. Kinda weird.

It therefore stands (in my mind, at least) that our Behat test suite should live in a completely independent project. This way all we need to change is the base implementation URL, and we can use one test suite to test multiple implementations.

That's the plan. Let's get to the implementation.

Creating a Behat JSON API Test Project

Once you've used a framework like Symfony, it's awfully tempting to use such a framework for every single project you create.

In truth, for my needs, using a big framework like Symfony even for small projects is often the simplest way to get started.

But in this case, because we really don't need much, we're going to roll a custom project, because why not?

mkdir behat-json-api-tester
cd behat-json-api-tester
composer init

We need two dependencies:

  • Behat
  • Imbo Behat API Extension

Follow the prompts, and you should end up with a composer.json file like this:

{
    "name": "codereviewvideos/behat-json-api-tester",
    "require": {
        "behat/behat": "^3.4",
        "imbo/behat-api-extension": "^2.1"
    },
    "authors": [
        {
            "name": "chris",
            "email": "chris@codereviewvideos.com"
        }
    ]
}

Ok, so we need Behat as our test framework.

The Imbo Behat API Extension gives us some ready-to-go feature step definitions, e.g.:

@When I request :path using HTTP :method

Which we will use like:

"Given I request "/album/1" using HTTP GET"

This simply means we don't need to write code to run our tests.

Given, When, Then, etc can all be used interchangeability. You change these to make your sentences make sense :)

As a quick heads up, imbo/behat-api-extension kindly includes Guzzle, so we can talk to our JSON API using HTTP.

Don't forget to composer install!

Configuring Behat

In order to use Behat we will need some additional files. Thankfully, we need not create these by hand:

php vendor/bin/behat --init

+d features - place your *.feature files here
+d features/bootstrap - place your context classes here
+f features/bootstrap/FeatureContext.php - place your definitions, transformations and hooks here

We're getting somewhere.

A new directory structure was created for us, within you'll find a new, empty FeatureContext.php.

This is enough to satisfy Behat to start running:

vendor/bin/behat

No scenarios
No steps
0m0.00s (7.81Mb)

Our project setup involved adding the Behat Web API Extension dependency. Behat will not magically just know that we want to use this extension. We must explicitly tell it.

touch behat.yml

The setup here pulls in the step definitions so our own Feature files can make use of all the pre-defined steps in the imbo/behat-api-extension:

# behat.yml

default:
    suites:
        default:
            contexts:
                - FeatureContext
                - Imbo\BehatApiExtension\Context\ApiContext
    extensions:
        Imbo\BehatApiExtension:
            apiClient:
                base_uri: http://127.0.0.1:8000

We're going to use the built in Symfony web server initially, which runs by default on 127.0.0.1:8000. We will change this as needed later on.

In order for Behat to do useful stuff for us, we now need to define our JSON API Features.

Testing JSON APIs with Behat

We got a glimpse of what a Behat Feature looks like a little earlier. First, let's create our two new Feature files:

touch features/healthcheck.feature
touch features/album.feature

The health check feature is a very basic test to prove, on a fundamental level, that our API is up. The feature is simple, you can optionally skip this, but for a quick debug, it's a useful thing to have:

# features/healthcheck.feature

Feature: To ensure the API is responding in a simple manner

  In order to offer a working product
  As a conscientious software developer
  I need to ensure my JSON API is functioning

  Scenario: Basic healthcheck
    Given I request "/ping" using HTTP GET
    Then the response code is 200
    And the response body is:
    """
    "pong"
    """

You can edit the above to be whatever you'd like. It's not true JSON. But then it need not be, as this is purely for your own test purposes.

# features/album.feature

Feature: Provide a consistent standard JSON API endpoint

  In order to build interchangeable front ends
  As a JSON API developer
  I need to allow Create, Read, Update, and Delete functionality

  Background:
    Given there are Albums with the following details:
      | title                              | track_count | release_date              |
      | some fake album name               | 12          | 2020-01-08T00:00:00+00:00 |
      | another great album                | 9           | 2019-01-07T23:22:21+00:00 |
      | now that's what I call Album vol 2 | 23          | 2018-02-06T11:10:09+00:00 |

  Scenario: Can get a single Album
    Given I request "/album/1" using HTTP GET
    Then the response code is 200
    And the response body contains JSON:
    """
    {
      "id": 1,
      "title": "some fake album name",
      "track_count": 12,
      "release_date": "2020-01-08T00:00:00+00:00"
    }
    """

  Scenario: Can get a collection of Albums
    Given I request "/album" using HTTP GET
    Then the response code is 200
    And the response body contains JSON:
    """
    [
      {
        "id": 1,
        "title": "some fake album name",
        "track_count": 12,
        "release_date": "2020-01-08T00:00:00+00:00"
      },
      {
        "id": 2,
        "title": "another great album",
        "track_count": 9,
        "release_date": "2019-01-07T23:22:21+00:00"
      },
      {
        "id": 3,
        "title": "now that's what I call Album vol 2",
        "track_count": 23,
        "release_date": "2018-02-06T11:10:09+00:00"
      }
    ]
    """

  Scenario: Can add a new Album
    Given the request body is:
      """
      {
        "title": "Awesome new Album",
        "track_count": 7,
        "release_date": "2030-12-05T01:02:03+00:00"
      }
      """
    When I request "/album" using HTTP POST
    Then the response code is 201

  Scenario: Can update an existing Album - PUT
    Given the request body is:
      """
      {
        "title": "Renamed an album",
        "track_count": 9,
        "release_date": "2019-01-07T23:22:21+00:00"
      }
      """
    When I request "/album/2" using HTTP PUT
    Then the response code is 204

  Scenario: Can update an existing Album - PATCH
    Given the request body is:
      """
      {
        "track_count": 10
      }
      """
    When I request "/album/2" using HTTP PATCH
    Then the response code is 204

  Scenario: Can delete an Album
    Given I request "/album/3" using HTTP GET
    Then the response code is 200
    When I request "/album/3" using HTTP DELETE
    Then the response code is 204
    When I request "/album/3" using HTTP GET
    Then the response code is 404

This feature describes, from a very high level, how our JSON API should behave.

We aren't testing exactly how things work internally. We are relying on the systems output, and a variety of HTTP response codes to ensure the system is behaving as expected.

You may disagree with some of this. The PATCH scenario is somewhat contentious. Feel free to change, alter, adapt, or complete disregard as you see fit.

Background System Setup

Our Album.feature has some expectations.

The obvious ones are that we have some JSON API setup somewhere that can handle GET, POST, PUT, PATCH, and DELETE requests.

Less obvious is that we kinda expect some data to already be there when we run our tests. All these Albums don't just appear out of nowhere. We must explicitly tell Behat how they came to be.

As ever, there's a ton of ways we could achieve this. Typically you would have your Behat tests right alongside your project code so, for example, you could use your Symfony / Doctrine integration to create any entities and save these off to the database as required.

Another approach might to use raw SQL files, and mysqldump or pg_dump to restore these files every time you run your test suite.

We're going to have a combination of both MySQL and Postgres databases to deal with. I don't like working with raw SQL files as they are not super friendly.

Instead, we're going to - rather contentiously - dogfood our own JSON API in order to bootstrap our system.

This is somewhat unintuitive. In order to set up our database we will need to POST in new data to our API so that our tests can run....

But, yet we won't have an endpoint for POST'ing data in to begin with, so how can we run our tests?

It's a little chicken and egg. You're welcome to go about this process however you see fit. I'm not saying this is the best way, it's just a way.

Anyway, it gives us a good place to start - our first POST request.

Episodes

# Title Duration
1 What will our JSON API actually do? 08:46
2 What needs to be in our Database for our Tests to work? 12:32
3 Cleaning up after each Test Run 02:40
4 Docker makes for Easy Databases 09:01
5 Healthcheck [Raw Symfony 4] 07:53
6 Send in JSON data using POST [Raw Symfony 4] 05:33
7 Keep your data nice and tidy using Symfony's Form [Raw Symfony 4] 10:48
8 Validating incoming JSON [Raw Symfony 4] 08:26
9 Nicer error messages [Raw Symfony 4] 06:23
10 GET'ting data from our Symfony 4 API [Raw Symfony 4] 08:11
11 GET'ting a collection of Albums [Raw Symfony 4] 01:50
12 Update existing Albums with PUT [Raw Symfony 4] 05:00
13 Upsetting Purists with PATCH [Raw Symfony 4] 02:39
14 Hitting DELETE [Raw Symfony 4] 02:11
15 How to open your API to the outside world with CORS [Raw Symfony 4] 07:48
16 Getting Setup with Symfony 4 and FOSRESTBundle [FOSRESTBundle] 09:11
17 Healthcheck [FOSRESTBundle] 06:14
18 Handling POST requests [FOSRESTBundle] 08:31
19 Saving POST data to the database [FOSRESTBundle] 09:44
20 Work with XML, or JSON, or Both [FOSRESTBundle] 04:31
21 Going far, then Too Far with the ViewResponseListener [FOSRESTBundle] 03:19
22 GET'ting data from your Symfony 4 API [FOSRESTBundle] 05:58
23 GET'ting a Collection of data from your Symfony 4 API [FOSRESTBundle] 01:27
24 Updating with PUT [FOSRESTBundle] 02:58
25 Partially Updating with PATCH [FOSRESTBundle] 02:15
26 DELETE'ing Albums [FOSRESTBundle] 01:27
27 Handling Errors [FOSRESTBundle] 08:58
28 Introducing the API Platform [API Platform] 08:19
29 The Entry Point [API Platform] 04:30
30 The Context [API Platform] 05:52
31 Healthcheck - Custom Endpoint [API Platform] 05:17
32 Starting with POST [API Platform] 07:08
33 Creating Entities with the Schema Generator [API Platform] 07:38
34 Defining A Custom POST Route [API Platform] 07:31
35 Finishing POST [API Platform] 06:29
36 GET'ting One Resource [API Platform] 02:50
37 GET'ting Multiple Resources [API Platform] 02:59
38 PUT to Update Existing Data [API Platform] 02:19
39 DELETE to Remove Data [API Platform] 01:15
40 No One Likes Errors [API Platform] 03:28