Code Review Videos > Broken Link Checker > TypeScript > Handling Bad Requests

Handling Bad Requests

If only life were as simple as covering the happy path in our software and we’re done. Unfortunately, back in the real world, the work truly begins when you’re done with the happy path and now you need to handle all the other stuff that can (read: will) go wrong.

The core part of our URL Checker is following redirects, allowing us to visualise the path a click takes from requesting URL A, and actually ending up on URL D, after passing through URL’s B & C.

We will get to the redirection journey shortly, but the next easier problem to tackle is HTTP status codes > 399. In other words, if the returned status code is anything like 400, 401, 403, 418, or 500, we want to exit and return whatever we have found so far.

Bad Request Test

We already know what a bad request looks like, as we have previously modelled a bad request.

At its most basic, our bad request will look like this:

const badRequest = {
  status: -1,
  ok: false,
  redirected: false,
  headers: {},
};Code language: TypeScript (typescript)

But in the event of, say, a 400, then we could add some extra information to this object.

Here’s an immediately passing test that covers such a scenario:

  test("should return a request representation if not a 2xx response but doesn't have a valid nextLocation", async () => {
    const fetcherSpy = jest.spyOn(Fetcher, "fetcher").mockResolvedValueOnce({
      ok: false,
      url: validUrl,
      status: 400,
      statusText: "Bad request",
      headers: { x: "y" },
    });

    expect(await visit(validUrl)).toEqual([
      {
        ok: false,
        url: validUrl,
        status: 400,
        statusText: "Bad request",
        headers: { x: "y" },
        redirected: false,
      },
    ]);

    expect(fetcherSpy).toHaveBeenCalledTimes(1);
    expect(fetcherSpy).toHaveBeenCalledWith(validUrl);
  });Code language: TypeScript (typescript)

Why does this pass immediately?

Because our code currently doesn’t much beyond taking the response from the fetcher and, as we saw previously, it adds in the hardcoded redirected: false key / value pair, and we are done.

Really Bad Request Test

The previous test was worthwhile to add, but our implementation already covered the outcome.

What happens if you make a request for a valid looking URL that turns out to fail in a way that doesn’t return a response?

Well, let’s try it via the command line way of running our little app, and see what happens:

import { VisitedURL } from "./types";
import { badRequest } from "./bad-request";
import { isValidUrl } from "./is-valid-url";
import { fetcher } from "./fetcher";

export const visit = async (
  href: string,
  requests: VisitedURL[] = []
): Promise<VisitedURL[]> => {
  if (!isValidUrl(href)) {
    return [
      ...requests,
      {
        ...badRequest,
        url: href,
        statusText: `Invalid URL`,
      },
    ];
  }

  const hrefToUrl = new URL(href);

  if (!["http:", "https:"].includes(hrefToUrl.protocol)) {
    return [
      ...requests,
      {
        ...badRequest,
        url: href,
        statusText: `Unsupported protocol: "${hrefToUrl.protocol}"`,
      },
    ];
  }

  const result = await fetcher("https://a.bad.url");
  console.log(result);

  return [];
};
Code language: TypeScript (typescript)

Just like last time, we will need to compile our TypeScript code to JavaScript, so Node can run it from the command line. You can do this in several ways, but most simply:

// from your project root dir
node ./node_modules/.bin/tscCode language: Shell Session (shell)

This should, if you are following along with the way I’ve been working, spit out lots of JavaScript files in your ./dist directory.

You can then call the index.js file using Node:

➜  link-visitor-typescript git:(main) ✗ node dist/v2/index.js       

(node:2212110) ExperimentalWarning: The Fetch API is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11118:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  cause: Error: getaddrinfo ENOTFOUND a.bad.url
      at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26) {
    errno: -3008,
    code: 'ENOTFOUND',
    syscall: 'getaddrinfo',
    hostname: 'a.bad.url'
  }
}Code language: Shell Session (shell)

Again, not the prettiest thing when dumped out as code. Here it is as it looks in my terminal:

The key thing to note here is that we don’t see the empty array dumped out to the console. That means the function didn’t complete.

In other words, when fetch fails, it throws. And in our case, we do not catch the error, so it bubbles up and bombs out.

Let’s add a new unit test to cover what we actually want to happen in this scenario.

Catch Me If You Can

Here’s the new test:

  test("should return a bad request representation if the fetch process fails", async () => {
    const fetcherSpy = jest.spyOn(Fetcher, "fetcher");
    const badUrl = "http://something.bad";

    expect(await visit(badUrl)).toEqual([
      {
        status: -1,
        ok: false,
        redirected: false,
        headers: {},
        url: badUrl,
        statusText: "An unhandled error occurred: XXXXX",
      },
    ]);

    expect(fetcherSpy).toHaveBeenCalledTimes(1);
    expect(fetcherSpy).toHaveBeenCalledWith(badUrl);
  });Code language: TypeScript (typescript)

We’re going to wrap our call to the fetcher in a try / catch block, and then take the error message from any thrown exception, and output that in our statusText message.

But for now, let’s assume we don’t know what the error message might be (the XXXXX placeholder), and see what our test spits out:

Not very helpful.

With Jest you can drill down a little bit, by pressing p in watch mode:

This is one of the nicest features of Jest, in my opinion.

There’s a bunch of options here, but the one we will use is p and then type in the relative path to our file:

I don’t need the full path, nor the full filename. So long as it’s unique enough, it will run just the test file I want.

Again, not super useful.

Uncovered Lines

We can now see the specific test that fails. And weirdly, fetcher shows as having uncovered lines, as does is-valid-url.

What this means is that, based on the narrow subset of files we just tested, these lines are uncovered. If we run the full test suite (press p again, then the Escape key), then we get back to running all tests and see our coverage is actually 100%. Quirky.

Narrowing Down The Test Failure

In our test code we have three assertions:

  test("should return a bad request representation if the fetch process fails", async () => {
    const fetcherSpy = jest.spyOn(Fetcher, "fetcher");
    const badUrl = "http://something.bad";

    expect(await visit(badUrl)).toEqual([
      {
        status: -1,
        ok: false,
        redirected: false,
        headers: {},
        url: badUrl,
        statusText: "An unhandled error occurred: .",
      },
    ]);

    expect(fetcherSpy).toHaveBeenCalledTimes(1);
    expect(fetcherSpy).toHaveBeenCalledWith(badUrl);
  });
Code language: TypeScript (typescript)

Our test fails.

But which of the three assertions fails?

Well, I don’t know a better way than trial and error for this step, unfortunately. By which I mean the easiest way to narrow this down is by commenting out the assertions and seeing what error messages you get.

A tip here is to mark the test as .only, so only this test runs:

  test.only("should return a bad request representation if the fetch process fails", async () => {
    const fetcherSpy = jest.spyOn(Fetcher, "fetcher");
    const badUrl = "http://something.bad";

    expect(await visit(badUrl)).toEqual([
      {
        status: -1,
        ok: false,
        redirected: false,
        headers: {},
        url: badUrl,
        statusText: "An unhandled error occurred: .",
      },
    ]);

    // expect(fetcherSpy).toHaveBeenCalledTimes(1);
    // expect(fetcherSpy).toHaveBeenCalledWith(badUrl);
  });
Code language: TypeScript (typescript)

Which gives:

And if we flip this around:

  test.only("should return a bad request representation if the fetch process fails", async () => {
    const fetcherSpy = jest.spyOn(Fetcher, "fetcher");
    const badUrl = "http://something.bad";

    // expect(await visit(badUrl)).toEqual([
    //   {
    //     status: -1,
    //     ok: false,
    //     redirected: false,
    //     headers: {},
    //     url: badUrl,
    //     statusText: "An unhandled error occurred: .",
    //   },
    // ]);

    expect(fetcherSpy).toHaveBeenCalledTimes(1);
    expect(fetcherSpy).toHaveBeenCalledWith(badUrl);
  });Code language: TypeScript (typescript)

Then we get a different error:

From this we can deduce that the test fails at the first assertion.

The reason being that our fetcher calls fetch with a bad url, which throws and we do not catch the error.

OK, let’s fix that by wrapping our call in a try / catch block:

import { VisitedURL } from "./types";
import { badRequest } from "./bad-request";
import { isValidUrl } from "./is-valid-url";
import { fetcher } from "./fetcher";

export const visit = async (
  href: string,
  requests: VisitedURL[] = []
): Promise<VisitedURL[]> => {
  if (!isValidUrl(href)) {
    return [
      ...requests,
      {
        ...badRequest,
        url: href,
        statusText: `Invalid URL`,
      },
    ];
  }

  const hrefToUrl = new URL(href);

  if (!["http:", "https:"].includes(hrefToUrl.protocol)) {
    return [
      ...requests,
      {
        ...badRequest,
        url: href,
        statusText: `Unsupported protocol: "${hrefToUrl.protocol}"`,
      },
    ];
  }

  try {
    const result = await fetcher(href);

    return [
      {
        ...result,
        redirected: false,
      },
    ];
  } catch (e) {
    return [
      ...requests,
      {
        ...badRequest,
        url: href,
        statusText: `An unhandled error occurred: ${e}`,
      },
    ];
  }
};
Code language: TypeScript (typescript)

What we have done here is retain the same logic if everything goes to plan. That’s lines 35-42.

But if anything goes wrong, we use the same behaviour as previously covered and append a customised statusText property with our wrapped error message.

In this code the e variable is being implicitly coerced from an Error to a string. You might think, why not just type the e to be an Error and then explicitly call .toString()? The reason is because that’s harder than it ought to be in TypeScript.

Anyway, with the try / catch block added to the implementation we now get further in our test output:

What this means is we need to update our test with the expected message, and we should be good:

  test("should return a bad request representation if the fetch process fails", async () => {
    const fetcherSpy = jest.spyOn(Fetcher, "fetcher");
    const badUrl = "http://something.bad";

    expect(await visit(badUrl)).toEqual([
      {
        status: -1,
        ok: false,
        redirected: false,
        headers: {},
        url: badUrl,
        statusText: "An unhandled error occurred: TypeError: fetch failed",
      },
    ]);

    expect(fetcherSpy).toHaveBeenCalledTimes(1);
    expect(fetcherSpy).toHaveBeenCalledWith(badUrl);
  });
Code language: TypeScript (typescript)

Be sure to remove swap back from test.only( to test( on your opening line if you had narrowed this down.

Anyway, we should be passing now.

Let’s keep going and look at links that redirect.

Previous
Next

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.