Efficient JSON Logging for Node.js.

And how using the Winston transport library is going to help you do your Node.js logging

For those who love so much JavaScript that they want to run it on browsers and servers: Node.js is the solution. As Wikipedia states: “Node.js is a packaged compilation of Google’s V8 JavaScript engine, the libuv platform abstraction layer, and a core library, which is itself primarily written in JavaScript.” So it comes as no surprise that Node.js has become a widespread language over the last years. And so comes the issue of Node.js logging.

We found that sending your Node.js logs through TCP with the Winston library and Logstash is the most efficient way to get value out of them, as it allows you to log in Json and use already well developed and rich components. So we’re going to cover here the different steps to do just this.

What is Winston and how to use it?

Winston is a great multi-transport async logging library which comes with a lot of features and available add-ons. It handles exceptions and comes with a set of pre-defined error levels you can actually extend.

Of course Winston is available through NPM. So first edit your package.json file to add the dependency and the logstash add-on that we’ll use later:

npm install --save winston

This command updates your package.json with the latest winston version.

{

"name": ...

...

 "dependencies": {

                 ...

                 "winston": "1.0.0",

                 "winston-logstash": "*"

                 ...

 }

}

Now open a test .js file and paste the following code:

var winston = require('winston');

var logger = new (winston.Logger)({

    transports: [

       new (winston.transports.File)({

           name: 'info-file',

           filename: 'filelog-info.log',

           level: 'info'

           })

    ]

});

logger.log('info', 'Hello simple log!');

logger.info('Hello log with metas',{color: 'blue' });

Adding metas to any log is extremely simple. You just need to provide any object with all the key-value pairs you want to see in the log event. These metas can be hostnames, usernames, customers, metrics or any information that help you troubleshoot and understand what happens in your Node.js application.

Execute this code and check the content of the filelog-info.log file:

{"level":"info","message":"Hello simple log!","timestamp":"2015-04-23T16:52:05.337Z"}

{"color":"blue","level":"info","message":"Hello log with metas","timestamp":"2015-04-23T16:52:05.339Z"}

See? Winston already took care of logging everything in JSON. You now have simple, structured encoding and logs which information is much easier to use and filter later down the road.

Centralize your JSON logs with Logstash

There are many solutions to centralize your logs. We find that sending them to your platform over TCP is a good trade-off in terms of performance and reliability, so we’re going to focus on TCP.
Your logs can also easily be sent through your favorite connector, whether it be Logstash or Fluentd, to your log management software. We’re going to focus here on using the most common one: Logstash transport.

First of all, add the corresponding dependency:

npm install --save winston-logstash

Then, executing this piece of code and check the incoming data in your platform:

var winston = require('winston');

require('winston-logstash');

var logger = new (winston.Logger)({

  transports: [

       new (winston.transports.Logstash)({

           port: 10514,

           host: 'endpoint.example.io',

           max_connect_retries: 1,

           node_name: 'my node name',

       })

  ]

});

logger.log('info', 'Hello simple log!');

logger.info('Hello log with metas',{color: 'blue', geo: {country:'France',city: "Paris"} });

Don’t forget to get your endpoint running! And if you want to make a simple hook, I recommend you to run the following command:

nc -l 10514

Secure the communication with a touch of SSL

If you want to transmit your logs through SSL, the logstash add-on actually comes with everything needed. To do so, change the target port to your ssl one – for instance 10515 – and enable SSL as follows:

var winston = require('winston');

require('winston-logstash');

var logger = new (winston.Logger)({

  transports: [

       new (winston.transports.Logstash)({

       port: 10515,

       ssl_enable: true,

       host: 'endpoint.example.io',

       max_connect_retries: -1,

       node_name: 'my node name',

       })

  ]

});

logger.log('info', 'Hello simple log!');

logger.info('Hello log with metas',{color: 'blue', geo: {country:'France',city: "Paris"} });

Wrapping up

Winston is a great library as it directly provides you with all the features you need to build up your Node.js logging strategy and support whatever decisions you will make as a team of developers.

A couple of advice to make Node.js logging useful when you need it:

• Always give your logger a name that corresponds to the functionality or service you’re trying to deliver.
• Log a lot at DEBUG level and accurately in INFO, WARNING and FATAL levels as these are the log levels you’ll be searching for in your production environments.
• Start small and try to log what matters first instead of being comprehensive. Adding missing elements later on when the issue rises in team discussions does not take long.
• Use metas! Adding context to any of your logs will help you quickly filter over users, customers or any business centric attribute.

I’ll leave you there and hope you enjoy Winston!