Jmeter Results Visualisation Using InfluxDB Grafana

1.     Introduction

This article provides the details of setting up InflxDB and Grafana for performance test monitoring using Jmeter. Running Jmeter tests in command line mode provides a benefit of running the performance tests without consuming much system resources, also, provide the ability to scale the user load on any machine. However, one has to wait until the end of the test to view the results. InfluxDB and Grafana provide a feature to store and analyse time-series data respectively and could be used to real-time monitoring of the.

Installing InfluxDB and Grafana is a bit tedious task, but Blazemeter recently provided a white paper to install the InfluxDB and Grafana using Docker.

Blazemeter provides all the necessary setup files, which could be downloaded straight to your computer and can be used with Docker.

Setting up InfluxDB and Grafana involves the following steps

·        Installation of Docker

·        Installation of InfluxDB and Grafana

·        Setting up Grafana and Datasource

·        Setting up Jmeter Dashboard

2.     Installation of Docker

Download the Docker community edition from https://www.docker.com/community-edition and follow the installation instructions.

Open the PowerShell or command prompt and type “docker --version” to verify the successful installation of Docker. Given command returns the installed version details of Docker.

3.     Installation of InfluxDB and Grafana using Docker

Once you have Docker and Docker Compose successfully installed and up and running. Next step would be to download the Docker file created by Blazemeter using following link

https://github.com/BushnevYuri/DockerGrafanaInfluxKit

Place the downloaded unzipped files to a folder and then follow the following steps

1.      Open the console

2.      Navigate to “DockerGrafanaInfluxKit-master” folder

3.      Run the command: > docker-compose up

If you see the following screen then the installation is successful

if you open http://localhost:3000 after the installation you should see your Grafana:

You can login using the default user ‘admin’ and ‘admin’ password. Also, if you go to the Grafana data sources configuration (Left top menu button -> Data Sources) you will see that the installed Grafana already has an inbuilt configuration with an InfluxDB instance that we are running in a separate docker container:

To be 100% sure that the integration works correctly, you can click on the ‘InfluxDB’ data source, scroll down to the end of the page and click on ‘Save and Test’. If you got ‘Data source is working’ which means your lightweight monitoring environment is ready and verified!

Following details will be required to set up the performance test monitoring using Jmeter

·        Host address: localhost or 127.0.0.1

·        Database port: 8086

·        Database name: influx

·        Database username: admin

·        Database password: admin

4.     Setting up Jmeter

To setup the Jmeter to push the data to Influx DB you will require the Backend Listener in you test plan, however to use the Jmeter dashboard provided by Grafana you will require to include the “JMeter-InfluxDB-Writer 1.2” jar file into the /lib/ext directory of your JMeter installation. Jar file can be downloaded using following link. https://github.com/NovaTecConsulting/JMeter-InfluxDB-Writer/releases/tag/v-1.2  

Restart your Jmeter and include the backend listener with the following values.

5.     Setting up Jmeter Dashboard

Next step is to setup the Jmeter dashboard. Simplest way to do this is to use the Jmeter Dashboard and you can download this Grafana template in json format using link https://grafana.com/dashboards/1152

After that, open Grafana and instead of the new dashboard creation go to:

Menu -> Dashboards -> Import and choose “Download JSON”, specifying the data source that we have just created for that example:

After that, by clicking on “Import” we will automatically create outstanding JMeter monitoring dashboard

JavaScript: String And Math Functions

Introduction of JAVAScript as an alternate scripting languages in Loadrunner version 12.50  gives the flexibility to the Performance Test Engineers to leverage the capabilities of existing JAVAScript functions to code various custom logics. 

I am going to describe couple of pre built JAVAScript String and Math functions which will make your life bit easier. 

Date.js
Working with dates are always tricky. So many dots, slashes and formats. Datejs is a stunning library for simple to complex date functions. It can parse dates like: “Next thursday”, “+2 years” and all formats like 2009.01.08, 12/6/2001 etc.

Sylvester
This is a JavaScript library for doing vector and matrix math easily, without using a lot of loops and getting lost between arrays. It includes classes for modelling vectors and matrices in any number of dimensions and for modelling infinite lines and planes in 3D space.

Pretty Date 
A smart JavaScript solution for displaying dates within the past month in a prettier and more user-friendly way. It displays dates relative to the current time; for example, “Yesterday,” “3 hours ago,” etc.

XRegExp
Regular expressions can be already used in JavaScript with the RegExp object.XRegExp adds more power to RegExp with features that will be part of the browsers of tomorrow (according to proposals for ECMAScript 4 – ES4). Using the library, RegExp objects can be cached and reused, modifiers can be added to existing RegExp objects and more.

JavaScript URL Library
A library for handling and manipulating URLs more easily. It is possible to reach every part of a URL as a string and modify it when needed. This URL library is very new but already works as mentioned.

ASP .NET Application Monitoring

This article describes the basic monitoring required to drill down any ASP .NET related performance issues. It's always good to have more performance data than not enough, especially when you experience a problem that is not easily reproduced. This article will describe some necessary Performance Counters and logs required to be looked into while analysing ASP .NET application's performance issues.

Monitoring Performance Counters:
There are many performance counters that are useful for monitoring performance of ASP .NET application. Below are the few important ones. 

Processor(_Total)\% Processor Time

Process(aspnet_wp)\% Processor Time

Process(aspnet_wp)\Private Bytes

Process(aspnet_wp)\Virtual Bytes

Process(aspnet_wp)\Handle Count

Microsoft® .NET CLR Exceptions\# Exceps thrown / sec

ASP.NET\Application Restarts

ASP.NET\Requests Rejected

ASP.NET\Worker Process Restarts (not applicable to IIS 6.0)

Memory\Available Mbytes

Web Service\Current Connections

Web Service\ISAPI Extension Requests/sec

Monitoring Logs:

Event Log:

It is critical to monitor the event log for messages from ASP.NET and Microsoft Internet Information Server (IIS). ASP.NET writes messages to the application log, for example, each time the aspnet_wp worker process terminates IIS 6.0 writes messages to both the application and/or system logs, for example, each time the w3wp worker process reports itself unhealthy or crashes. It is quite easy to write a .NET application that reads the application log and filters out messages from ASP.NET and IIS, and fires an alert (sends e-mail or dials a pager) if necessary.

W3C and HTTPERR Logs:

First, enable W3C logging for IIS through the Internet Information Services (IIS) Manager. This log can be configured to include various data about the requests, such as the URI, status code, and so on. Scan the log for error codes such as 404 Not Found, and take action to correct links, if necessary. It also contains substatus code in the log and is useful for debugging. IIS uses substatus codes to indentify specific problems. For example, 404.2 indicates that the ISAPI extension handling the request is locked down. 

New for IIS 6.0, malformed or bad requests and requests that fail to be served by an Application Pool are logged to the HTTPERR log by HTTP.SYS, the kernel-mode driver for handling HTTP requests. Each entry includes the URL and a brief description of the error.

Check the HTTPERR log for rejected requests. Requests are rejected by HTTP.SYS when the kernel request queue is exceeded, and when the application is taken offline by the Rapid Fail Protection feature. When the first issue occurs, the URL is logged with the message QueueFull, and when the second occurs, the message is AppOffline. Check the HTTPERR log for requests lost due to a worker process crash or hang. When this occurs the URL will be logged with the message,Connection_Abandoned_By_AppPool, for each in-flight request. An in-flight request is one that was sent to a worker process for processing, but did not complete before the crash or hang.