SPE 101 – Refreshing The Basics – Part V

This is Part V of a V part series.

Introduction – The intention of this series from the start has been to cover some of the most fundamental aspects of Systems Performance Engineering. The fundamentals of Systems Performance Engineering have been dealt with in great detail at the SPE Fundamentals section here at Practical Performance Analyst and this series of articles aims to introduce you the reader to some of the most common terminology that you would come across in your daily professional lives when dealing with IT systems from a performance standpoint. You are currently reading part V of a V part series and links to the earlier posts in the series are available at the start of this article. We would encourage readers who want to dig further into the basics of Systems Performance Engineering to visit the SPE Fundamentals section here at Practical Performance Analyst.

In the first part of this series we started looking at the fundamentals of System Performance Engineering by diving into Response Time (R), What makes up Response Time, How Response Time Is Measured and Time to First Byte or TTFB.  In the second part of the series we looked at Think Time (Z), Service Time (S) and the simple equation for Service Demand (S=B/C).  In the third part of the series we will look at the following fundamental performance quantities like Queuing Time, Wait Time, Throughput, Completions, Busy Time and Queue Length. In the fourth part of this series we took a look at the definition of System Utilization, the definition of Little’s Law and finally the application of Little’s Law in a few real life situations. So, let’s get started with Part V where we wrap up the series by looking at the Response Time Model and ErlangC model.

Response Time Model: The Response Time model is a simple way of calculating the overall Queue Length and Queuing Time for a given system for a given set of workload conditions. The Response time Model is generally used for back of the envelope calculations and to get a sense of direction. We wouldn’t recommend using Response Time models to make investment or major architectural design decisions. For more detailed performance modelling when validating architectural decisions or investment decisions we would recommend investing in building thorough Queuing Theory models.

To be able to use the Response Time model at minimum you would to know two of the three variables mentioned below:

  • Uavg – Average System Utilization
  • X – System Throughput
  • St – Service Time

M is always assumed to be known since it’s the number of CPU’s (also called Servers when referring to performance models) within the system. This is a design assumption and is assumed to be a known quantity.

  • M – Number of Servers

Let’s take a look at all of the equations and then look at a real world example:

  • Uavg = [ X * St ] / M ……………..  [ Uavg = Average Utilization,  X = Throughput, St = Service Time, M = Number of Servers ]
  • Rt = St / [ 1 –  UavgM ]………..  [ Rt = CPU Response Time, Uavg = Average Utilization,  St = Service Time, M = Number of Servers ]
  • Rt = Qt + St ………………………..  [ St = Service Time, Rt = Response Time, Qt = Queuing Time ]
  • Qt = Rt – St………………………  [ Rt = CPU Response Time, Qt = Queuing Time, St = Service Time ]
  • Q = X * Qt …………………………..  [ Q = Queue Length, Qt = Queuing Time, St = Service Time ]

Example: The best way to understand these laws is to take an example and work our way through the various equations:

Let’s assume we observe a system with 4 CPU’s for a period of 180 seconds out of which the system has been busy for 90 seconds. For the duration of observation there were 5 successfully completed transactions. Let’s use the very basic Response Time theory to do some quick back of the envelope transactions and understand what the Queuing Times would be and how long would the Queue have been on the system.

  • T = 180 seconds [ Total Time of Observation ]
  • B = 90 seconds [ Busy Time ]
  • C = 5 [ Completions ]
  • M = 4 [ Number of CPU’s or Servers ]
  • We know :
    • St = B / C ………………. [ St = Service Time ]
    • Therefore : St = 90 / 5 = 18 seconds
  • We also know:
    • X = C / T …………………….[ X = Throughput ]
    • X = 5 / 180 = 0.028 TPS
  • We also know:
    • Uavg = [ X * St ] / M ………………..[ Uavg = Average Utilization ]
    • Uavg = [ 0.028 * 18 ] / 4 =       0.126
  • We also know :
    • Rt = St / [ 1 –  UavgM ] …………. [ Rt = CPU Response Time ]
    • Rt = 18 / [ 1 – 0.126 4]
    • Rt = 18 / [ 1 – 0.504 ]
    • Rt = 36.2 seconds
  • We also know:
    • Rt = Qt + St …………. [ Qt = Queuing Time ]
    • Qt = Rt – St
    • Qt = 36.2 – 18 = 18 seconds
  • We also know:
    • Q = X * Qt   [ Q = Queue Length ]
    • Q = 0.028 * 18
    • Q = 0.504

Thus, using Response Time theory we’ve figured out that the actual Queue size is 0.0504.

What is Erlang C – Telecommunications traffic has traditionally been measured in dimensionless units called Erlangs, named after the Danish telephone engineer, Agner Krarup Erlang. The number of Erlangs of traffic is equal to the average number of calls received per unit of time, multiplied by the average duration of a call.  If you have a system in which calls are queued rather than dropped, then you can use the Erlang C formula to calculate the probability that a call is queued and the average call waiting time, aka the Average Speed of Answer (ASA). To use the Erlang C formula, you need to know the traffic level and the number of trunks or lines that are available to take calls.

The Erlang C formula has been traditionally used to determine how many phone lines are needed to attain a desired probability of waiting. Erlang C has over a period of time been used to identify the capacity requirements for different business scenarios e.g. call center agents required, ticket collectors required, etc. This section provides a view of the ErlangC formulae which you could apply to any system including the IT systems you come on contact with on a daily basis.

Erlang C Model for a CPU Sub System: 

  • Xq = Xsys / Qn ……………………..[ Xq = Arrival Rate at queue, Xsys = System Arrival Rate or Total Arrival Rate, Qn = Number of Queues in the System (should be 1 for CPU systems)]
  • Uavg =  [ Xq * St ] / M …………….[ Xq = Arrival Rate at queue, St = Service Time, M = Number of Servers In the System, Uavg = Average Utilization across CPU’s]
  • Ec = ErlangC (M, St, Xq) ………..[ Xq = Arrival Rate at queue, St = Service Time, M = Number of Servers In the System ]
  • ErlangC (M, St, Xq) =  { [ (M St Xq) M ] / m ! }   /   { (1 – M St) Sigma(m-1) k=0  [ (M St Xq) k / k ! ] + [ (M St Xq) m ] / m ! }
  • Qt = Ec St / [ M (1 – Uavg) ] ……….[ Ec = ErlangC Constant, St = Service Time, M = Number of Servers In the System, Uavg = Average Utilization across CPU’s, Qt = Queueing Time]
  • Q = Xq * Qt …………………………..[ Xq = Arrival Rate at queue, Qt = Queueing Time, Q = Queue Length ]
  • Rcpu = Qt + St ………………………..[ Rcpu = CPU Response Time, Qt = Queueing Time, St = Service Time ]

Erlang C Model for a IO Sub System: 

  • Xq = Xsys / Qn ……………………..[ Xq = Arrival Rate at queue, Xsys = System Arrival Rate or Total Arrival Rate, Qn = Number of Queues in the System]
  • Uavg =  [ Xq * St ] / M …………….[ Xq = Arrival Rate at queue, St = Service Time, M = Number of Servers In the System (Should be 1 for an IO Sub System), Uavg = Average Utilization across CPU’s]
  • Ec = ErlangC (M, St, Xq) ………..[ Xq = Arrival Rate at queue, St = Service Time, M = Number of Servers In the System (Should be 1 for an IO Sub System)]
  • ErlangC (M, St, Xq) =  { [ (M St Xq) M ] / m ! }   /   { (1 – M St) Sigma k=0 (m-1) [ (M St Xq)k / k ! ] + [ (M St Xq) m ] / m ! }
  • Qt = Ec St / [ M (1 – Uavg) ] ……….[ Ec = ErlangC Constant, St = Service Time, M = Number of Servers In the System (Should be 1 for an IO Sub System), Uavg = Average Utilization across CPU’s, Qt = Queuing Time]
  • Q = Xq * Qt …………………………..[ Xq = Arrival Rate at queue, Qt = Queuing Time, Q = Queue Length ]
  • Rcpu = Qt + St ………………………..[ Rcpu = CPU Response Time, Qt = Queuing Time, St = Service Time ]

Modelling Solution: Applying some of the above formulae to model system behavior and forecast system performance can be tedious. Those interested in making use of the various modelling techniques covered in this series might want to head over and check out VisualizeIT. VisualizeIT offers access to a bunch of Analytical Models, Statistical Models and Simulation Mcropped-visualize_it_logo__transparent_090415.pngodels for purposes of Visualization, Modelling & Forecasting. Access to all the Analytical (Mathematical) models is free. You can access the VisualizeIT website here and the VisualizeIT modelling solution here –VisualizeIT.

Conclusion – We hope you have enjoyed this five part series on the basics of Systems Performance Engineering as much as we did putting it together. We hope we’ve taken you a bit further in your understanding of the basics concepts around Systems Performance. We would encourage readers who want to dig further into the basics of Systems Performance Engineering to visit the SPE Fundamentals section here at Practical Performance Analyst.


Trevor Warren (Linked In) loves hacking open source, designing innovative solutions and building trevor_warrencommunities. Trevor is inquisitive by nature, loves asking questions and some times does get into trouble for doing so. He’s passionate about certain things in life and building solutions that have the ability to impact people’s lives in a positive manner is one of them. He believes that he can change the world and is doing the little he can to change it in his own little ways. When not hacking open source, building new products, writing content for Practical Performance Analyst, dreaming up new concepts or building castles in the air, you can catch-him bird spotting (watching planes fly above his house).

Practical Performance Analyst as an Open Body Of Knowledge on Systems Performance Engineering (SPE) built + maintained by Trevor with the support of his army of volunteer elves (PPA Volunteers). You can reach trevor at –  trevor at practical performance analyst dot com. The views expressed on this web site are his own.

Related Posts

  • SPE 101 – Refreshing The Basics – Part IVSPE 101 – Refreshing The Basics – Part IV This is Part IV of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I You can read the second post here - SPE 101 - Refreshing The Basics - Part II You can read the third post here - SPE 101 - Refreshing The Basics - Part […]
  • SPE 101 – Refreshing The Basics – Part IIISPE 101 – Refreshing The Basics – Part III This is Part III of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I You can read the second post here - SPE 101 - Refreshing The Basics - Part II Introduction - The intention of this series from the start has been to cover some of […]
  • SPE 101 – Refreshing The Basics – Part IISPE 101 – Refreshing The Basics – Part II This is Part II of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I Introduction - It’s been a while since we have touched upon the basics of SPE (Systems Performance Engineering) here at Practical Performance Analyst. The SPE Fundamentals […]
  • SPE 101 – Refreshing The Basics – Part ISPE 101 – Refreshing The Basics – Part I It’s been a while since we have touched upon the basics here at Practical Performance Analyst. Everyone can do with a refresher once in a while it so why not take a quick peek at some of the basic definitions of SPE (Systems Performance Engineering) quantities that most of us deal with […]
  • Sudeep Dutt

    Hi Trevor,
    I believe the calulations are wrong for Rt as the formula should be Rt = St/ [1 – (Uavg)^M] and not Rt = St/ [1 – Uavg * M]. Thus, Rt = 18 Seconds & Qt = 0 Seconds & Q (Length) = 0

    • Sudeep – Thanks for pointing it out mate. Much appreciated.