SPE 101 – Refreshing The Basics – Part III

This is Part III of a V part series.

Introduction – The intention of this series from the start has been to cover some of the most fundamental aspects of Systems Performance Engineering. The fundamentals of Systems Performance Engineering have been dealt with in great detail at the SPE Fundamentals section here at Practical Performance Analyst and this series of articles aims to introduce you the reader to some of the most common terminology that you would come across in your daily lives when dealing with IT systems from a performance standpoint. As we’ve mentioned before this is part III of a V part series. We would encourage readers who want to dig further into the basics of Systems Performance Engineering to visit the SPE Fundamentals section here at Practical Performance Analyst.

In the first part of this series we started looking at the fundamentals of System Performance Engineering by diving into Response Time (R), What makes up Response Time, How Response Time Is Measured and Time to First Byte or TTFB.  In the second part of the series we looked at Think Time (Z), Service Time (S) and the simple equation for Service Demand (S=B/C).  In this part of the series we will look at the following fundamental performance quantities –

  • Queuing Time
  • Wait Time
  • Throughput
  • Completions
  • Busy Time
  • Queue Length

Queuing Time or Qt – Queuing Time is defined as the time spent waiting in a queue for access to computing resources. Queuing Time for a given business transaction can be also be defined as the End to End Queuing Time for access of any of the following physical resources i.e.

  • Queuing Time for access to Disk Resource
  • Queuing Time for access to CPU Resources
  • Queuing Time for access to Application Server Resources
  • Queuing Time for access to Database Server Resources

Let’s revisit our basic equation for Response Time:

  • Rt = Wt + St …………………….  [ Wt = Wait Time, St = Service Time ]  also written as
  • Rt = Qt + St  …………………….  [ Wt = Wait Time, Qt = Queuing Time ]

Queuing Time [ Wt or Qt ] for a given application business transaction doesn’t remain constant and is a variable that can change based on the congestion within the system and movement of congestion points within the given system. Longer queuing times will translate into more dropped connections and surely a lot more irate customers.

response_time_1_191215

Performance Testing is generally conducted to identify potential system bottlenecks and understand the extent of queuing the system can take before the overall system performance starts degrading. Queuing on a system is inevitable and one can only invest that much of money to eliminate queuing based on choice of application designs and deployment architectures.

Wait Time or Wt [Same as Queuing Time] – Same as Queuing Time

Busy Time or Bt – Busy Time is simply defined as the time duration for which the server resource i.e. cpu, mem, etc. was busy doing work. Service Time (S) which we’ve seen earlier is also written as S = B/C or Busy Time / Number Of Completions.

Transactional Completions or C – Transactional Completions is simply defined as the number of successful transactions that were processed within a given time period. Completions are denoted by C.

Transactional Arrivals or A – Transactional Arrivals are simply defined as the number of requests arriving at the server for processing. Transactional Arrivals are denoted by A.

Queue Length or Q – The Queue Length is simply defined as the total number of requests waiting in the queue.

Average Number of Servers (CPU’s, Cores, Threads) or M – M is used to designate the Average number of CPU’s, Cores or Threads on a given Server.

Number of Users within the System or N – The number of active or concurrent users performing a given action within a system is denoted by N.

Throughput or X – Throughput is defined as the rate of completions for a given system. It’s also defined as the total completions per unit time e.g. Transactions/Sec.

Throughput_100216

The equation for throughput is defined as:

X = C / T …………………[C = Completions in time T, T = Total time of observation]

Example: Let’s assume we observe a system for a period of 180 seconds (3 minutes) during which there are 5 customers who successfully exit the system. The system can be described as:

  • Completions or C = 5
  • Time Period of Observation or T = 180s
  • Throughput or X = C / T = 5 / 180 = 0.03 Transactions/Sec

There is also one other interesting relationship between Maximum Throughput of a system and the Average Value of Service Time. It’s denoted by the following equation:

Xmax <= 1 / Stavg …[Xmax= Max Value of Throughput possible, Stavg = Avg Value of Service Time ]

The above equation is one of the most fundamental and important equations with regards to system performance, also called the bottleneck equation. The bottleneck equation basically states that the Maximum Throughput is inversely proportional to the Average (System/Component) Service Time. Using the above equation you can key in the Average Service Time for your bottleneck resource (resource or component within the system that has the highest Service Time) within your system and obtain the maximum possible throughput the system can achieve.

Example: Let’s assume for the same system above the highest Service Time across the various system components is 3s. Thus Xmax <= 1/3s = 0.33 Transactions/Sec. This particular system can never achieve a throughput more than 0.33 Transactions/Sec as long as the biggest Service time within the system has a Service Time of 3s.

The above equation comes in very handy when validating performance testing results. The bottleneck equation is also very frequently used to perform back of the envelope calculations for bottleneck and throughput analysis.

Conclusion – Queuing Time (Qt), Wait Time (Wt), Throughput (X), Completions (C), Busy Time (B) and Queue Length (Q) are some of the fundamental quantities that make up most of the basic equations that Performance Engineers use very frequently to understand and model their system Performance. Use of these quantities and equations in general is not limited to IT systems but also any system that you come across in real life i.e. stores with check out counters where you stand for service, cinema counters where you queue up to be served, etc. We would highly encourage you to build on these concepts by visiting the Systems Performance Engineering (SPE) Fundamentals section at Practical Performance Analyst SPE Fundamentals.


Trevor Warren (Linked In) loves hacking open source, designing innovative solutions and building trevor_warrencommunities. Trevor is inquisitive by nature, loves asking questions and some times does get into trouble for doing so. He’s passionate about certain things in life and building solutions that have the ability to impact people’s lives in a positive manner is one of them. He believes that he can change the world and is doing the little he can to change it in his own little ways. When not hacking open source, building new products, writing content for Practical Performance Analyst, dreaming up new concepts or building castles in the air, you can catch-him bird spotting (watching planes fly above his house).

Practical Performance Analyst as an Open Body Of Knowledge on Systems Performance Engineering (SPE) built + maintained by Trevor with the support of his army of volunteer elves (PPA Volunteers). You can reach trevor at –  trevor at practical performance analyst dot com. The views expressed on this web site are his own.

Related Posts

  • SPE 101 – Refreshing The Basics – Part IVSPE 101 – Refreshing The Basics – Part IV This is Part IV of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I You can read the second post here - SPE 101 - Refreshing The Basics - Part II You can read the third post here - SPE 101 - Refreshing The Basics - Part […]
  • SPE 101 – Refreshing The Basics – Part VSPE 101 – Refreshing The Basics – Part V This is Part V of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I You can read the second post here - SPE 101 - Refreshing The Basics - Part II You can read the third post here - SPE 101 - Refreshing The Basics - Part III You […]
  • SPE 101 – Refreshing The Basics – Part IISPE 101 – Refreshing The Basics – Part II This is Part II of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I Introduction - It’s been a while since we have touched upon the basics of SPE (Systems Performance Engineering) here at Practical Performance Analyst. The SPE Fundamentals […]
  • SPE 101 – Refreshing The Basics – Part ISPE 101 – Refreshing The Basics – Part I It’s been a while since we have touched upon the basics here at Practical Performance Analyst. Everyone can do with a refresher once in a while it so why not take a quick peek at some of the basic definitions of SPE (Systems Performance Engineering) quantities that most of us deal with […]