One left over issue is that if we visit some URLs (not all, just some), then our function doesn’t always return in a timely fashion.
For example, if we call:
node dist/v2/index.js https://codereviewvideos.com/typescript-tupleCode language: Shell Session (shell)Then whilst we do see the expected output – 3 unique GET request / responses in our array – the function appears to hang, rather than returning back to the command line.
If you try this, and you get stuck, ctrl + c will exit out.
However, if you wait for ~30 seconds, it will eventually return to the command line by itself.
Why is this?
Manual Redirects
It took me a while to track this down when I first wrote the code. In hindsight it kinda seems obvious:
const { url, status, statusText, ok, headers } = await fetch(href, {
redirect: "manual",
});
Code language: TypeScript (typescript)Way back, we set the redirect option to be manual.
The redirect: "manual" option in the fetch function tells our code to not automatically follow redirects. Instead, the fetch function will return a response with the redirect status code, allowing us to manually handle the redirect if desired.
If we don’t do this then getting all the request / response outcomes for every visited link in the chain is not possible.
But it also has the knock on effect that we need to manually terminate / abort the pending request, or it may hang around waiting to automatically time out.
Essentially each our visit calls is an isolated / standalone request. When there are redirects thrown in, if we don’t explicitly handle them, the fetch function seems to sit there shrugging its shoulders saying hey, what do you want to do with this? Should I follow it? Should I give up? Hello? Hello?!
Manually Aborting Requests
A working solution to this problem seems to be aborting every request immediately after we have the response. Seems logical.
import { getNextLocation } from "./get-next-location";
export const fetcher = async (href: string) => {
const controller = new AbortController();
const { signal } = controller;
const { url, status, statusText, ok, headers } = await fetch(href, {
redirect: "manual",
signal,
});
controller.abort();
const headersObject = Object.fromEntries(headers);
const nextLocation = getNextLocation(url, headersObject);
return { url, status, statusText, ok, headers: headersObject, nextLocation };
};
Code language: TypeScript (typescript)You can read up on the abort signal on the MDN docs.
We can use the controller.abort() method to cancel the fetch request.
It should already be complete as we destructured the url, status, etc from the response, right?
But those pesky redirects, they seem to hang around due to our desire to manual-ly handle them.
Before writing a test for that – because there’s going to be a painful part with regards to the highlighted line – we could manually test whether it resolves the problem or not.
The key thing is it exits – immediately – back to the command line. It’s worth running it through several times to ensure it does always behave this way. But me showing you the same screenshot over and over is … erm, not fun.
Extract The Abort Controller Setup
Whenever we have to integrate with something that we don’t control, automated testing can become tricky.
We saw this way back when working with fetch.
And the problem rears its ugly head again now, in the form of:
const controller = new AbortController();Code language: TypeScript (typescript)The problem is that we are tightly coupling our implementation to a very specific way of doing things.
It would be far better to rely on some kind of abstraction.
And this is actually really easy to achieve. We just need to extract the instantiation of the new AbortController logic, and move it to its own function. That we way depend on our own function, which is nice and easy to mock. This won’t result in a perfect decoupling, but it should be enough for the complexity of application we are building.
// abort-controller.spec.ts
import { createAbortController } from "./abort-controller";
describe("abort controller", () => {
test("should return an AbortController instance", () => {
expect(createAbortController()).toBeInstanceOf(AbortController);
});
});Code language: TypeScript (typescript)And the implementation:
export const createAbortController = () => new AbortController();Code language: TypeScript (typescript)If we had a more complex application it would probably be far more preferable to define our own interface and only return the very specific bits we actually need – such as the signal.
But as above, our application doesn’t require that level of granularity, so I’d opt for not making things too complex.
Anyway, we can now use that logic:
import { getNextLocation } from "./get-next-location";
import {createAbortController} from "../abort-controller";
export const fetcher = async (href: string) => {
const controller = createAbortController();
const { url, status, statusText, ok, headers } = await fetch(href, {
redirect: "manual",
signal: controller.signal,
});
controller.abort();
const headersObject = Object.fromEntries(headers);
const nextLocation = getNextLocation(url, headersObject);
return { url, status, statusText, ok, headers: headersObject, nextLocation };
};
Code language: TypeScript (typescript)It’s still basically the same,
We should update our tests to take calling the abort signal into account:
import { fetcher } from "./fetcher";
import * as AbortController from "../abort-controller";
const fakeUrl = "http://some.fake.url";
describe("fetcher", () => {
afterEach(() => {
jest.resetAllMocks();
});
[
{
description: "empty headers",
givenHeaders: new Headers(),
expectedHeaders: {},
},
{
description: "with headers",
givenHeaders: new Headers({ some: "data" }),
expectedHeaders: { some: "data" },
},
].forEach(({ description, givenHeaders, expectedHeaders }) => {
test(`should call the native fetch function in the expected manner: ${description}`, async () => {
const mockAbortFn = jest.fn();
const fakeAbortSignal = "pretend abort signal code";
jest
.spyOn(AbortController, "createAbortController")
.mockImplementationOnce(
() =>
({
signal: fakeAbortSignal,
abort: mockAbortFn,
} as unknown as AbortController)
);
const fetchSpy = jest.spyOn(global, "fetch").mockResolvedValueOnce({
url: fakeUrl,
status: 200,
statusText: "something",
ok: true,
headers: givenHeaders,
} as unknown as Response);
const result = await fetcher(fakeUrl);
expect(fetchSpy).toHaveBeenCalledTimes(1);
expect(fetchSpy).toHaveBeenCalledWith(fakeUrl, {
redirect: "manual",
signal: fakeAbortSignal,
});
expect(result).toEqual({
ok: true,
status: 200,
statusText: "something",
url: "http://some.fake.url",
headers: expectedHeaders,
nextLocation: null,
});
expect(mockAbortFn).toHaveBeenCalledTimes(1);
});
});
test("should guess the next location if applicable", async () => {
const mockAbortFn = jest.fn();
const fakeAbortSignal = "pretend abort signal code";
const fakeLocationPath = "/a/b/c";
jest.spyOn(AbortController, "createAbortController").mockImplementationOnce(
() =>
({
signal: fakeAbortSignal,
abort: mockAbortFn,
} as unknown as AbortController)
);
jest.spyOn(global, "fetch").mockResolvedValueOnce({
url: fakeUrl,
status: 200,
statusText: "something",
ok: true,
headers: new Headers({ location: fakeLocationPath }),
} as unknown as Response);
const result = await fetcher(fakeUrl);
expect(result).toEqual({
ok: true,
status: 200,
statusText: "something",
url: "http://some.fake.url",
headers: { location: fakeLocationPath },
nextLocation: `${fakeUrl}${fakeLocationPath}`,
});
expect(mockAbortFn).toHaveBeenCalledTimes(1);
});
});
Code language: TypeScript (typescript)And so finally, all the tests pass and the code is complete.
That wraps up the TypeScript implementation.