SPE 101 – Refreshing The Basics – Part IV

This is Part IV of a V part series.

Introduction – The intention of this series from the start has been to cover some of the most fundamental aspects of Systems Performance Engineering. The fundamentals of Systems Performance Engineering have been dealt with in great detail at the SPE Fundamentals section here at Practical Performance Analyst and this series of articles aims to introduce you the reader to some of the most common terminology that you would come across in your daily professional lives when dealing with IT systems from a performance standpoint. As we’ve mentioned before this is part IV of a V part series and links to the earlier posts in the series are available at the start of this article. We would encourage readers who want to dig further into the basics of Systems Performance Engineering to visit the SPE Fundamentals section here at Practical Performance Analyst.

In the first part of this series we started looking at the fundamentals of System Performance Engineering by diving into Response Time (R), What makes up Response Time, How Response Time Is Measured and Time to First Byte or TTFB.  In the second part of the series we looked at Think Time (Z), Service Time (S) and the simple equation for Service Demand (S=B/C).  In the third part of the series we will look at the following fundamental performance quantities like Queuing Time, Wait Time, Throughput, Completions, Busy Time and Queue Length.

In the fourth part of this series we will dive into Utilization, then take a look at Little’s Law and finally look at the application of Little’s Law in a few real life situations. So, let’s get started……

Utilization or U – Utilization of any system is defined as the Ratio of Busy Time to Total Time of Observation. Utilization is a ratio that generally refers to how busy or free the resources within a given system are. Utilization can also be used to refer to how busy or free are components within a given system instead of the system as a whole.

utilization_210216

The following equations are used to describe Utilization:

U = Bt / T   …………………………. [ Bt = Busy Time, T = Total Observed Time Interval ]

The above equation describes utilization as the ratio of Busy Time to Total Observed Time.

Example: Let’s assume we are viewing a system for a period of 180 seconds during which it remains busy for 100 seconds. The Utilization of the system is calculated as follows:

U = 100/180 = 0.55

The system utilization is this 0.55 and is a ratio.

U = X * St   …………………………. [ X = Throughput, St = Service Time ]

The above equation describes Utilization as the product of Throughput and Service Time. As mentioned earlier the above equation can be applied to either the system as a whole or at a resource level within a given system.

Example: Let’s assume we are viewing a system for a period of 180 seconds during which it remains busy for 100 seconds. The Service Time for a transaction is 2s. The throughput of the system is calculated as follows:

U = X * St

X = U / St = 0.55 / 2 = 0.275 Transactions/Sec

Uavg =  [ X * St ] / M …………….. [ Uavg = Average Utilization, X = Throughput, St = Service Time, M = Average Number of Servers  ]

The above equation is very similar to the previous equation but with one subtle difference. On a system with multiple servers (CPU’s) the average utilization is obtained by dividing the overall system utilization by the total number of servers (CPU’s) present.

Little’s Law:  John Dutton Conant Little, is an Institute Professor at the Massachusetts Institute of Technology, best known for his result in operations research, Little’s law. Little’s law is truly amazing in its simplicity and can be used to describe the most complex of systems including resources within those systems.

arrivals_and_Departures_210216Let’s take for example the system described above. The notations used include:

  • N – Number of users present within the system
  • C – Number of completions
  • X – Throughput or Rate of Departure
  • A – Number of Arrivals
  • λ – Rate of Arrival
  • Rt – Time spent by Customers within the system

Little’s Law basically states that the long-term average number of customers in a stable system N is equal to the long-term average effective arrival rate, λ, multiplied by the average time a customer spends in the system, W or Rt, or expressed algebraically:

N = λ * Rt  ………………..  [ N = Number of Users in the System, Rt = Response Time, λ = Arrival Rate ]

Little’s Law can also be stated as:

N = Rt * X  ………………..  [ N = Number of Users in the System, Rt = Response Time, X = Throughput ]

For a system where Zt (Think Time is Non Zero) Little’s Law can be stated as:

N =  [Rt + Zt] * X  ………………..  [ N = Number of Users in the System, Rt = Response Time, Zt = Think Time, X = Throughput ]

Applying Little’s Law:  Now that we’ve just defined Little’s Law let’s take a look at where we could apply Little’s Law and benefit from a better understanding of our systems including user behavior on our existing systems.

lineup_210216

Little’s Law can be applied to the following scenarios:

  • Model Performance Testing Workload
  • Validate Performance Testing Results
  • Validate the Sizing Models that your Infrastructure guys have come up with
  • Validate the Non Functional Requirements that your Architects have recommended

Let’s take a look at a few examples and apply Little’s Law to a real world system:

Example – Determine Concurrent Users Using Little’s Law: A system at peak processes 8000 Transactions/Hour with an average response time of ~5s per transaction. The average Think Time per user is ~30s. What is the number of concurrent users on the system?

Throughput or X = 8000/3600 = 2.22 TPS

We know: N =  [Rt + Zt] * X  ………..  [ N = Number of Users in the System, Rt = Response Time, Zt = Think Time, X = Throughput ]

Applying the equation:

N = [ Rt + Z ] * X = [ 5 + 30 ] * 2.22 = 77.7

The above system has approximately 77 users at peak driving 8000 Transactions/Hour.

Example – Validate System Performance Testing Results: The outcome of a performance test is as follows:

  • Peak User Concurrency = 1000
  • Average Transactional Response Time = 5s
  • Average User Think Time = 40s
  • Peak Transactional Throughput ( as recorded by the tool) = 500 TPS

We know: N =  [Rt + Zt] * X  ………..  [ N = Number of Users in the System, Rt = Response Time, Zt = Think Time, X = Throughput ]

Applying the equation:

X = N / [ Rt + Z ] = 1000 / [ 5 + 40 ] = 22.22 TPS

You know from the above results, something is really wrong. Little’s Law suggests that the actual TPS for the above system should be ~22 TPS but the Performance Testing results show transactional throughput as 500 TPS. Could this be errors in the Performance Testing results due to poorly written scripts? If your scripts are not smart enough to catch all the relevant errors, the Performance Testing tool will include the errors as part of the true transactional throughput.

The above two examples are meant to give you some familiarity with Little’s Law and help you apply Little’s Law to model Performance Testing workload including validating your Performance Testing results.

Conclusion – In this series of SPE 101 Refresh the basics we’ve looked at the definition of System Utilization, the basics of Little’s Law (Part of Operational Theory) and the application of Little’s Law. The fundamentals of Systems Performance Engineering have been dealt with in great detail at the SPE Fundamentals section here at Practical Performance Analyst and this series of articles aims to introduce you the reader to some of the most common terminology that you would come across in your daily professional lives when dealing with IT systems from a performance standpoint. We hope you’ve enjoyed this piece on the series and until next time, Au Revoir!!.


Trevor Warren (Linked In) loves hacking open source, designing innovative solutions and building trevor_warrencommunities. Trevor is inquisitive by nature, loves asking questions and some times does get into trouble for doing so. He’s passionate about certain things in life and building solutions that have the ability to impact people’s lives in a positive manner is one of them. He believes that he can change the world and is doing the little he can to change it in his own little ways. When not hacking open source, building new products, writing content for Practical Performance Analyst, dreaming up new concepts or building castles in the air, you can catch-him bird spotting (watching planes fly above his house).

Practical Performance Analyst as an Open Body Of Knowledge on Systems Performance Engineering (SPE) built + maintained by Trevor with the support of his army of volunteer elves (PPA Volunteers). You can reach trevor at –  trevor at practical performance analyst dot com. The views expressed on this web site are his own.

Related Posts

  • SPE 101 – Refreshing The Basics – Part VSPE 101 – Refreshing The Basics – Part V This is Part V of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I You can read the second post here - SPE 101 - Refreshing The Basics - Part II You can read the third post here - SPE 101 - Refreshing The Basics - Part III You […]
  • SPE 101 – Refreshing The Basics – Part IIISPE 101 – Refreshing The Basics – Part III This is Part III of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I You can read the second post here - SPE 101 - Refreshing The Basics - Part II Introduction - The intention of this series from the start has been to cover some of […]
  • SPE 101 – Refreshing The Basics – Part IISPE 101 – Refreshing The Basics – Part II This is Part II of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I Introduction - It’s been a while since we have touched upon the basics of SPE (Systems Performance Engineering) here at Practical Performance Analyst. The SPE Fundamentals […]
  • SPE 101 – Refreshing The Basics – Part ISPE 101 – Refreshing The Basics – Part I It’s been a while since we have touched upon the basics here at Practical Performance Analyst. Everyone can do with a refresher once in a while it so why not take a quick peek at some of the basic definitions of SPE (Systems Performance Engineering) quantities that most of us deal with […]