Throughput, Concurrency Tests and Importance of Think Time

​Introduction

Throughput and concurrency tests are something interesting load testing methodology and if used properly can drastically reduce testing time and costs.

In load/performance testing, throughput refers to the rate in which a web application can process raw data, full pages, or entire transactions.

With a “throughput test” you are aiming to stress the throughput of the web application. The throughput of a web application is usually measured by pages per second, objects/hits per second, or by transactions per minute.

When applied to the Web and web applications, concurrency is the measure of how many simultaneous user sessions are active on a web application at a given point of time.

In a “concurrency test” you’re trying to test how many concurrent user sessions a web application can support. In Reality Load, this is signified by the total number of virtual users that are active during a test.

At the basic level the only difference between a throughput test and a concurrency test is the length of the user sessions.

In the load testing, session length is controlled by the “think time” – a time delay that is added into test scripts to simulate the pacing of real users using a web application – within a script.

Importance of Think Time

People have to think about what they see, read it, make a decision, and then click. Each person will respond to the delivered page in a slightly different time frame. Think time is variable because real users have a large deviation in their ability to process information. Some are visual learners, some are analytical, some are contemplative, some are driving personalities,

This whole scenario can be understood by taking a simple example:

Suppose there is a script which has 3 transactions and each transaction takes 2 sec to complete so the entire script will complete in 6 secs and 1 vuser will take 12 sec to complete 2 iterations, If we add 2 sec think time after each transaction then 1 iteration will get completed in 12sec, so here we are now achieving only 1 iteration in 12 sec and we we need to achieve 2 iterations in 12 sec then we’ll have to take 2 Vusers.

To take this concept a little further, let’s take a look at the following example. Suppose we have a script (or transaction) that includes 3 pages, and each page takes approximately 2 seconds to load and 6 seconds for the entire transaction to complete.

A test with no or very little think time would be considered a throughput test. By adding “think time” to the test, you can maximize the number of concurrent users on the application, while achieving the same throughput. A test with significant “think time” added to the test can be considered a concurrency test.

Depending upon the application context, the think time would vary. Its not advisable to have the default think time for all applications under testing. For the user registration page containing 5 mandatory fields, the use might take 15-20 seconds on an average. Notice the words “15-20 seconds on an average”. You can’t say that all the users would take exactly a specific think time. It would definitely vary based on each of the users.

As we are interested to measure the response time of the transaction (user defined actions), care should be taken to place the think time outside the transaction points. (i.e.) the think time should not be placed between the start & end of the transaction.

We can simulate the load of large number of users though we run the test for less number of users. To illustrate, consider the following example. Five tests were conducted with 10 virtual users simultaneously accessing the application. In each of the test, the think time is the only metric which was changed. Observer, the Hits/sec generated from the client side & Throughput (the number of transactions passed /sec)

Note: This is just a example which i am using to make the concept more clear

No. Of Vusers	Think time in Sec	Hits/Sec	Trans/Sec	%CPU
10	5 to 10	23	.9	10-15%
10	2.5 to 5	48	1.8	15-20%
10	0	250	50	50-60%

Though the number of users hitting the server is same, the think time creates a huge impact on the server load. We need to understand the fact that decreasing the think time will increase the server load & vice versa. Always try to model the real time scenario & identify the performance figures. Never forget that we can play around with the server load just by changing the think time.

Why to use throughput and concurrency tests?

With a concurrency test you can achieve the same throughput, and the test will be more reflective of a real user experience with “think time”. This is true, and if time and money were unlimited, we probably wouldn’t need throughput tests.
However, time and money is often limited. Throughput tests are an effective means for reducing the cost and time of load testing. Looking back at our “think time” example, with a throughput test we can test the throughput of a web application with less virtual users than a concurrency test. Using less virtual users can shorten the time required to test.