Basic Setup: Azure Application Insights with Node.js & Express

I wanted to cover off a way I’ve been using Azure AppInsights in a NodeJS / Express application recently. But when I shared the post with a fellow developer for review, the feedback was that they had no idea what Azure Application Insights is, or how they might use it.

That got me to creating this post, which is a very basic set of setup steps required to get the Azure Application Insights SDK working in your NodeJS project.

You don’t need to be hosting your Node app on Azure, and you don’t need to be using Express.

But you do need an Azure account. The free one will do, with which you get $200 credit, or about £165 in credit for us folk across the pond.

What Is App Insights?

Azure Application Insights is a cloud-based application performance monitoring service provided by Microsoft Azure. It is designed to help developers and IT professionals gain insights into the performance, availability, and usage of their applications. Application Insights is part of the larger Azure Monitor service suite.

Key features of Azure Application Insights include:

1. Application Performance Monitoring (APM):
   • Request Tracking: Monitors and tracks incoming requests to your application, providing insights into response times and success rates.
   • Dependency Tracking: Identifies dependencies (like databases, external APIs) and monitors their performance.
2. Logging and Diagnostics:
   • Logging: Captures and stores log data, allowing you to diagnose issues and understand application behavior.
   • Exception Tracking: Monitors and logs exceptions in your application, helping you identify and fix errors.
3. User Analytics:
   • User Flows: Visualizes the paths that users take through your application, helping you understand user behavior.
   • User and Session Metrics: Tracks user engagement, retention, and other metrics.
4. Custom Event Tracking:
   • Custom Events: Enables tracking of custom events and metrics relevant to your application.
5. Live Metrics Stream:
   • Real-time Monitoring: Provides a live stream of telemetry data, allowing you to monitor your application in real-time.
6. Integration with Azure DevOps:
   • Integration: Easily integrates with Azure DevOps for continuous integration and deployment (CI/CD) workflows.
7. Alerts and Notifications:
   • Alert Rules: Sets up alert rules to be notified when specific conditions are met, such as a sudden increase in error rates.
8. Integration with Azure Services:
   • Azure Integration: Seamlessly integrates with other Azure services, facilitating a comprehensive monitoring solution.
9. Cross-Platform Support:
   • Language Support: Supports multiple programming languages, including .NET, Java, Node.js, Python, and more.

To use Azure Application Insights, you typically need to instrument your application by adding a small code snippet or using SDKs specific to your application’s programming language.

This instrumentation allows Application Insights to collect telemetry data and provide insights into your application’s performance and usage.

Do I Need To Be Hosted On Azure To Use AppInsights?

A fair question.

The answer is: no.

You do need an Azure account, but your application can run anywhere. It does not need to be running on, or hosted by Azure to use App Insights.

For example I log to Azure from a Dockerised setup on a self hosted server somewhere in Germany.

What Will We Be Setting Up?

We are going to set up a NodeJS Express app to log out requests and console data to the Azure cloud Logging platform.

This is a precursor to doing more interesting things, which will follow in future posts.

By the end of this post you will have a running Express app that you can browse to, and see data being logged out to both your local Express console, and the Azure Application Insights Log view.

This is particularly helpful when you have more than one server, for example in a Kubernetes cluster, and you need a centralised logging solution. It is also pretty helpful for smaller infrastructures (such as my own), where you want a central dashboard for seeing what’s happening, rather than having to SSH all over the show.

Setting Up Azure

I’m going to assume you have a Microsoft Azure account.

As I say, the free one will do. Getting one is a bit of a chore – they want an email, phone, and debit / credit card.

I will say this as a fair warning heads up: getting Azure to delete your credit card details is a massive pain in the ass. So if that makes you uncomfortable, get some credentials from work or something. It does make me uncomfortable, but I learned this the hard way (after spending ages on with their support people), so I want to share that right off the bat.

Still want to proceed?

Very good.

There are a whole bunch of steps you need to go through in order to setup App Insights for the first time inside Azure. Personally the first few times I had to use Azure I found it confusing, although not as confusing as AWS. Anyway, I figured I would share the exact step-by-step steps to complete in order to follow along.

I’m going to assume you have you Azure account set up, and are logged in.

Start by searching for App Insights from the top menu bar. It should pop out a box where Application Insights is listed under Services (the purple light bulb icon below):

When the Application Insights page loads for the first time, you won’t yet have any apps to show.

That makes sense.

Let’s create one:

Eughh.

Wordy form alert.

There’s two things you need to do here.

The first is create a Resource Group.

Click on the Create new link, and that pops up another box, where you can enter the name of your resource group.

Typically the name would be the environment, something like:

• prod-rg
• staging-rg
• dev-rg

Things like that. Where rg stands for Resource Group.

In this case it doesn’t much matter what you call it, as it’s just for experimentation and you can delete it later. I’ve gone with free-trial-resource-group.

Typically your Subscription would also be more specific. Perhaps you would have a subscription for all your dev / test resources, and then another separate subscription for prod. Here I have a rather unusually named Free Trial subscription :/

You will also need to give a name to the Instance you are creating. Personally I don’t have the most involved setup so I go with a single instance in a single resource group.

My Resource Group would be Prod, and my instance would be my App in prod, so maybe something like prod-app.

Personally I find it easier to keep the environments separate into their own Resource Groups, and that way I have one instance per resource group. That’s currently working for me, and I’d only make it more complex if I had too.

Finally the Workspace Details were auto-filled for me, so I left them as-is.

Once done, click Review + create.

At this point your config is validated.

I think that’s to ensure you don’t end up with two conflicting “things”, two resource groups of the same name, or two instances of the same name in the same resource group… that kind of thing.

Click Create when available.

You should see a pop up / toast message in the top right telling you that interesting things are happening.

Now, if you don’t see this, or you’re simply impatient, you can either refresh your browser window, or use the top bar to navigate back to App Insights (like we did at the start) and you should be taken to a “Deployment is in progress” page.

You have to be patient and wait for the deployment process to work. Who knows what magic is happening in the background?

After a while, you should see another pop up / toast message to say it all went fine.

If you click Go to resource and you still see the “Deployment is in progress” page, or you are very impatient, just hit the Refresh button – as shown below:

Once everything has deployed you should see the Overview page.

Lots to see here, but the most important part is the Essentials section at the top of the main pane.

In here you want the Connection String.

Hover over the Connection String and you can helpfully copy it to your clipboard.

You will need this in your Express app, so remember where this is.

We are done with the basic setup, but whilst we are here we might as well look at the page I tend to spend most of my time when actually using Azure App Insights on a day-to-day basis.

That’s the Logs view.

A Quick Look At Logs View

Whilst we’re on the App Insights Overview page, it’s the left hand side bar menu that you want.

In there you can see the Monitoring section, and under there, Logs.

Click on Logs:

This pops up another overlay box.

This can be quite handy at a future point as it should show historical queries you have crafted / run.

But for now it’s empty and is something we can ignore – so click the X in the top right to get rid of it.

This then shows you the Query view.

This is the most powerful and overwhelming part of the App Insights process.

It’s also what I want to focus on as the data it spits out can be super helpful for all kinds of reasons.

For now, just see that it is here, and remember how to access it. We will be back here shortly.

Setting Up Express To Log To Azure App Insights

In order to send data from Express to Azure App Insights, we unsurprisingly need an Express app up and running.

You might have one.

I’m going to assume you don’t, for the purposes of this post.

Not to worry, however, as creating one is easy enough.

Start by creating a new directory on your computer. I’m going to put this in the ~/Development/express-app-insights directory.

It doesn’t matter where it lives.

# check you have node installed
node -v
v20.5.0

# then make and enter your directory
mkdir ~/Development/express-app-insights
cd ~/Development/express-app-insightsCode language: Shell Session (shell)

Jolly good.

Now, create a new package.json file and add the following contents:

{
  "name": "typescript-express-starter",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "dev": "nodemon src/index.ts"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "applicationinsights": "^2.9.1",
    "express": "^4.18.2"
  },
  "devDependencies": {
    "@types/express": "^4.17.18",
    "@types/node": "^20.7.1",
    "nodemon": "^3.0.1",
    "prettier": "^3.0.3",
    "ts-node": "^10.9.1",
    "typescript": "^5.2.2"
  }
}Code language: JSON / JSON with Comments (json)

Then create a new tsconfig.json file:

{
  "compilerOptions": {
    "target": "es2016",
    "module": "commonjs",
    "rootDir": "./src",
    "outDir": "./dist",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true
  }
}Code language: JSON / JSON with Comments (json)

You don’t need to use TypeScript, but I wouldn’t start a new vanilla JS project these days, so I don’t do so here.

The last file we need is src/index.ts

import express from "express";

const app = express();
const port = process.env.PORT || 3000;

app.get("/", (req, res) => {
  res.status(200).json({ all: "working" });
});

app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});Code language: TypeScript (typescript)

That’s everything we need.

Let’s fire up the API.

# install the project dependencies

➜  npm i

added 450 packages, and audited 451 packages in 1s

45 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities


# start the server

➜  npm run dev

> typescript-express-starter@1.0.0 dev
> nodemon src/index.ts

[nodemon] 3.0.1
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: ts,json
[nodemon] starting `ts-node src/index.ts`
Server is running on port 3000Code language: Shell Session (shell)

You should now be able to visit http://0.0.0.0:3000 in your browser and see:

{"all":"working"}Code language: JSON / JSON with Comments (json)

Very good.

Adding App Insights

That’s our very basic Express app up and running.

Next let’s configure it to log out requests to Azure App Insights.

Here’s the changes we need:

import express from "express";
import * as appInsights from "applicationinsights";

const app = express();
const port = process.env.PORT || 3000;

appInsights
  .setup(
    "InstrumentationKey=xxx;IngestionEndpoint=https://...;LiveEndpoint=https://...",
  )
  .start();

app.get("/", (req, res) => {
  res.status(200).json({ all: "working" });
});

app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});
Code language: TypeScript (typescript)

Be sure to copy / paste your Connection String in from Azure as we covered above.

In a real world app, do not include the string like this. Use an environment variable. This is not secure, and also a very brittle way of configuring an application. This is only done like this for demo purposes.

We included applicationinsights as a dependency in package.json, that’s why this import is immediately available.

It’s actually a fairly chunky dependency:

However our API is running server side so that’s not that big of an issue here.

With that very minimal change, you should see the server immediately restart (thanks Nodemon!) and if you hit refresh on your http://0.0.0.0:3000 page a few times in your browser, you should have some logs to view in App Insights.

Basic App Insights Log View

OK, so we have set up Express with App Insights, run our server and hit the endpoint a few times to generate some request data.

How can we then see that data?

Well, back in Azure App Insights Query View, we need to craft our first query.

For that, all you need to do is type in requests, and then click Run.

All being well, you should see as many requests as you made appear in your query Results.

If you don’t see any results, here’s a couple of things to try:

1. Patience: it can take anywhere up to a couple of minutes (I believe 90 seconds is the default batch timer) to accumulate some logs and then ship them off to Azure. It is not instant.
2. Highlight your query: Sometimes I click Run and nothing seems to happen. If this is the case, I drag / highlight / select all the query text and then press Run again. This is quite useful on larger queries as it will only run the bits you highlight, allowing you to be quite granular.

Hopefully you are seeing some logs.

If you do, be sure to open one up and take a look at the detail view.

It’s pretty cool how much data we just logged, considering we didn’t explicitly log anything.

Log More Data

We will finish up here by updating our config to log console statements to Azure App Insights.

This will happen by default if you use bunyan or winston.

If you are just using console.log or console.debug etc, you need to tell your instance of App Insights to log these statements or it will ignore them.

Let’s update the TypeScript to do this:

import express from "express";
import * as appInsights from "applicationinsights";

const app = express();
const port = process.env.PORT || 3000;

appInsights
  .setup(
    "InstrumentationKey=xxx;IngestionEndpoint=https://...;LiveEndpoint=https://...",
  )
  .setAutoCollectConsole(true, true)
  .start();

app.get("/", (req, res) => {
  console.log("log");
  console.debug("debug");
  console.info("info");
  console.warn("warn");
  console.error("it broke", { some: "data" });
  res.status(200).json({ all: "working" });
});

app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});
Code language: JavaScript (javascript)

Again, Nodemon will ensure our server is automatically restarted and our TypeScript is converted to runnable JavaScript.

We just need to hit refresh once more on our http://0.0.0.0:3000 endpoint.

Now, update the query.

Change from requests to traces and hit Run.

If all goes to plan, you should see your logs output. Note the severity level for the warn and error logs is higher than log and info. Also note that console.debug was not sent.

All of this can be extremely useful in a real app.

You can start to see just how useful when you run a query like:

traces 
| where severityLevel > 1Code language: SQL (Structured Query Language) (sql)

I will leave that to you to play with in your own instance of App Insights.

OK, that’s us for the moment.

Wrapping Up

In this post we covered how to set up Azure App Insights for cloud logging purposes.

We then create a NodeJS Express API and added in the App Insights SDK and saw how it gives us general request logging without much configuration.

Then we added additional configuration to our App Insights SDK setup to allow console logging, and saw how we can send through console.log statements for adding extra data to help make use of our logs.

This was everything I needed to setup to actually show you more interesting things to do with Azure App Insights. That will follow in follow on posts.