Response Time Model: The Response Time model is a simple way of calculating the overall Queue Length and Queuing Time for a given system for a given set of workload conditions. The Response time Model is generally used for back of the envelope calculations and to get a sense of direction. We wouldn’t recommend using Response Time models to make investment or major architectural design decisions. For more detailed performance modelling when validating architectural decisions or investment decisions we would recommend investing in building thorough Queuing Theory models.

To be able to use the Response Time model at minimum you would need to know two of the three variables mentioned below:

  • Uavg – Average System Utilization
  • X – System Throughput
  • St – Service Time

M is always assumed to be known since it’s the number of CPU’s (also called Servers when referring to performance models) within the system. This is a design assumption and is assumed to be a known quantity.

  • M – Number of Servers

Let’s take a look at all of the equations and then look at a real world example:

  • Uavg = [ X * St ] / M ……………..  [ Uavg = Average Utilization,  X = Throughput, St = Service Time, M = Number of Servers ]
  • Rt-cpu = St / [ 1 –  (Uavg)M ]………..  [ Rt = CPU Response Time, Uavg = Average Utilization,  St = Service Time, M = Number of Servers ]
  • Rt = Qt + St ………………………..  [ St = Service Time, Rt = Response Time, Qt = Queuing Time ]
  • Qt = Rt – St………………………  [ Rt = CPU Response Time, Qt = Queuing Time, St= Service Time ]
  • Q = X * Qt …………………………..  [ Q = Queue Length, Qt = Queuing Time, St = Service Time ]

Example: The best way to understand these laws is to look at an example and work our way through the various equations:

Let’s assume we observe a system with 4 CPU’s for a period of 1 Hour or 3600 seconds out of which the system has been busy for 1000 seconds. For the duration of observation there were 500 successfully completed transactions. Let’s use the very basic Response Time theory to do some quick back of the envelope transactions and understand what the Queuing Times would be and how long would the Queue have been on the system.

  • T = 3600 seconds [ Total Time of Observation ]
  • B = 1000 seconds [ Busy Time ]
  • C = 500  [ Completions ]
  • M = 4 [ Number of CPU’s or Servers ]
  • We know from our fundamental equations :
    • St = B / C ………………. [ St = Service Time, B = Busy Time, C = Completions ]
    • Therefore : St = 1000 / 500 = 2 seconds
  • We also know from our fundamental equations:
    • X = C / T …………………….[ X = Throughput, C = Completions, T = Total Time of Observation ]
    • X = 500 / 3600 = 0.139 TPS
  • We also know:
    • Uavg = [ X * St ] / M  ………………..[ Uavg = Average Utilization ]
    • Uavg = [ 0.139 * 2 ] / 4 =  0.0695
    • The Average Utilization Per Processor is 0.0695
  • We also know :
    • Rt = St / [ 1 –  (Uavg)M ] …………. [ Rt = CPU Response Time ]
    • Rt = 2 / [ 1 – (0.0695)4]
    • Rt = 2 / [ 1 – 0.00002 ]
    • Rt = 2.000 seconds
  • We also know:
    • Rt = Qt + St …………. [ Qt = Queuing Time ]
    • Qt = Rt – St
    • Qt = 2.0 – 2.0  [ St = 2s, Rt = 2.0s)
    • Qt = 0 seconds
  • We also know:
    • Q = X * Qt   [ Q = Queue Length ]
    • Q = 0.139 * 0
    • Q = 0

Thus, using Response Time theory we’ve figured out that the actual Queue size Per CPU is 0 for the above set of variables.

We hope this example has illustrated the simplicity and power of Response Time Theory to estimate potential system bottlenecks. As we’ve mentioned before, only use Response Time models for quick back of the envelope calculations when you need to get a gut feel and direction. For a more detailed system performance model we would encourage you to consider using Erlang C model or Queuing Theory models which we’ve covered in other sections.

Modeling Systems & Forecasting Performance : To teach yourself the concepts of Performance Modeling & to experience how easy  Forecasting System Performance could be, please visit VisualizeIT.

  • Ramya

    Good Article. I agree that Response Time model analysis can be considered as preliminary step towards Queuing Theory principles. For the given input parameters, Queue length can be derived and potential bottlenecks can be identified. How this can be used to prediction analysis ? I mean to derive St / U / Qt / QL for next given TPS value. Can you please explain this.

    • Good question Ramya. As long as your Service Time (S) is known and remains constant along with your infrastructure configuration i.e. No Of Servers/CPU’s (M), you can gradually keep increasing the value of Throughput (Xavg) and work out the impact on the system i.e. Uavg.

      Also, have you taken a look at VisualizeIT? It offers this and many more analytical models for free.

      This is a topic that’s pretty easy to understand once you have experience including those who use it regularly but un-fortunately the majority of people out there don’t have the opportunity to use it regularly and don’t quite manage to get their head around the basics.

      You might even want to write a few articles around modelling using VisualizeIT to help other reader out.

  • Sudeep Dutt

    I believe the formula Rt = St / (1 – Uavg * M) is wrong. I think it should be Rt = St / (1 – (Uavg)^M).

    • Excellent stuff Sudeep. Thanks for picking this up..:)

      If you are able to spare sometime and are keen to volunteer across PPA please drop me a note at trevor at practical performance analyst dot com.

    • I’ve updated the models above. Have a read, review the calculations and drop me a note if you find anything else. Also, see my other note.

Thank You For Visiting & See You Soon

Practical Performance Analyst - NewsLetter

Signup and Stay In Touch

Signup to the Mailing List here at Practical Performance Analyst and Stay In Touch. We promise not to SPAM you.