SPE 101 – Refreshing The Basics – Part I

It’s been a while since we have touched upon the basics here at Practical Performance Analyst. Everyone can do with a refresher once in a while it so why not take a quick peek at some of the basic definitions of SPE (Systems Performance Engineering) quantities that most of us deal with in our day jobs across technology.

As I walk through each of these basic quantities I will help you understand their relevance from a performance engineering standpoint. Obtaining a good understanding of these definitions will also help you ask the right questions when working on a performance or capacity related challenge.

If you are interested in going into the depths of SPE (Systems Performance Engineering) we would suggest you dive into the SPE Fundamentals section here at Practical Performance Analyst – Click Here.

Let’s start with the most fundamental of all “Response Times”.

Response Time or Rt – Response Time is one of the most fundamental quantities in Performance Engineering. Response Time is defined as the Total Elapsed time between submission of a request and receipt of the response. Response Times can be measured at different tiers for a given application i.e. Web Server Response Time, Application Server Response Time, Database Server Response Time, etc. Response Times can also be measured within a given tier across the various system components i.e. CPU Response Time, Memory Access Response, Disk Access Response Time, Network Transfer Response Time, etc. Response Times have traditionally been measured for critical business transactions during a performance test using Diagnostics tools or Performance Testing tools. Response Times can also be measured for Single User Performance Tests using plugins like Firebug on Firefox or Developer Tools on Chrome (Which is totally awesome in the way it breaks down response times for content downloaded over the wire).

response_time_1_191215However, Response Times by-itself mean little and cannot provide anyone a good view of what the system performance looks like. It is recommended that one always consider Response Times in conjunction with other Performance Quantities i.e. System Utilization, User Concurrency, Application Throughput, Network Bandwidth Utilization, expected System SLA’s, etc. to understand what the true picture of Performance really is while keeping mind the cost of achieving that performance. Now that we’ve introduced the concept of Response Time let’s take the next step and see what the different types of Response Times are and where are these different Response Times measured.

The basic Response Time Equations are written as:

  • Rt = Wt + St …………………….  [ Wt = Wait Time, St = Service Time ]  also written as
  • Rt = Qt + St  …………………….  [ Wt = Wait Time, Qt = Queuing Time ]

So what are some of the challenges capturing Response Times – Applications these days are quite complex and consist of complex workloads (i.e. Multiple transaction types with different transaction rates) and interact with numerous different internal and external application components using Web Services/SOAP or custom protocols.  Measuring Response Times for such applications has traditionally been easy at the Business Transaction level using standard Performance Testing tools.

response_time_2_191215

But when it comes to measuring similar Response at each of the various application tiers (i.e. Web Server Tier, Application Server Tier, Database Server Tier, etc.) you run into challenges and things get further complicated if you need to capture response times for each of the workload components at the various systems components i.e. Response Time of the Order Submit Transaction at the CPU of the Application Server, Response Time of the Order Submit Transaction at the Disk Subsystem for Database Server.

  • End User Response Time = Web Server + App Server + EAI Server + DB Server
  • The above statement can also be written as: Rt = r1t + r2t + r3t + r4t

Now one can further break down response time at any tier across the following components:

  • r1t or Total Web Server Response Time = Time spent Queuing at Disk + Time spent on Disk IOPS + Time spent Queuing for CPU + Time spent at CPU
  • Similarly for r2t, r3t and r4

It should now be apparent that response times might be a simple metric that gets bandied about a lot when referring to performance but the intricacies of what’s involved is amazing and the level of detail you could go to completely depends on the nature of the analysis you intend to perform and the question we are trying to answer.

The point we are trying to make here is that Rt can be abstracted down to any level depending on the nature of the performance problem you are trying to solve. If all you are concerned about is the overall End User Performance for a given transaction you don’t have to go about measuring time spent by the transaction at each of the 10 Disks across the storage subsystem.

response_time_3_191215.png

Understanding Response Time

Measuring Response Times externally at the Business Transaction Tier using off the shelf Performance Testing tools or just a simple browser plugin like Firebug works really well. But when it comes to measuring Response Times for the Web Server Tier, Application Server Tier or the Database Server tier things get really complicated. This is simply because todays Performance Testing tools do not have the ability to go into the application and collect Response Times for each of the tiers.

You could use traditional Diagnostics tools which sit inside the Application Container (.Net or Java) to capture response times across the various tiers but that wouldn’t completely solve the problem either because Diagnostics tools only work on Application Servers i.e. .Net and Java, they don’t all necessarily work with Web Servers and other proprietary applications that you might be connecting to. So while Diagnostics tools can give you an idea of the Response Times of each of the individual method calls between the Application Server and Database Server you won’t be able to correlate the Response Times across the various external application tiers and definitely not with the Performance Testing tool.

There are a few solutions to the problem. One is to use transaction tracing tools that stamp packets with a code as the flow through the network. These agents sitting on the various servers (Web, Application, Database, etc.) need to have a good understanding of the application protocol along with support for the Operating System. The transaction tracing tool will then collect these statistics across the Web Servers, Application Servers, Database Servers, Messaging Servers, Web Services Servers, etc. and send them back to a central host that will provide an view of the Response Times across the various tiers. But try correlating this information with the results from the Performance Test and you’ll hit another brick wall. The second solution is to simply timestamp the message as it flows through the stack. The challenge with this approach is the fact that time stamping code takes time and very often you won’t have access to the source code of all the applications that you need to performance test.

Some of the newer Diagnostics tools i.e. New Relic and AppDynamics are changing the game and provide good view of End to End view of response times from the App Server through the Messaging Bus down to the Database server and even other 3rd Party applications.

Time to First Byte or TTFB – Another important Performance Quantity associated with Response Time is – Time To First Buffer or TTFB in short. Time To First Buffer is the Elapsed Time between submission of the request and receipt of the first response from the server. TTFB like Response Times can be measured either at the Application Tier level or within an Application Tier at any of system components i.e. CPU, Memory, Disk, Network, etc. The Time To First Buffer is an indication of how heavy the processing for a given workload is or even an indication of how long does it take to open up a connection to a particular resource for the first time due to various buffering requirements. Time to First Buffer is used a lot by Performance Testing teams and is also used to optimize the way applications deliver their payload to customers. Time To First Buffer gains a lot of relevant on applications where the rendering times are significant.

TTFB_1_121215Time To First Byte (TTFB)

Let’s take a look at an example to understand the relevance of Time To First Buffer.

Let’s say the user has submitted an Order Submit request at time t=0s. Let’s assume that the overall response time for the transaction is 8s, but the client received the first response at t=0.5s while rest of the response time is split across downloading of the rest of the components on the page, blocking time due to Javascripts including client GUI engine rendering the payload on the browser. The TTFB or Time to First buffer in this case is 0.5 or 500ms. TTFB is an important metric used to understand the initial response time for an application and the ability of a system to respond to a users request. However, use TTFB (Time To First Buffer) by itself does not do much. You would ideally want to use Time To First Buffer (TTFB) along with overall system or transaction Response Times, System Utilization, System Throughput, etc. to understand the behavior of a given system.

If you are interested in going into the depths of SPE (Systems Performance Engineering) we would suggest you dive into the SPE Fundamentals section here at Practical Performance Analyst – Click Here.


Trevor Warren (Linked In) loves hacking open source, designing innovative solutions and building trevor_warrencommunities. Trevor is inquisitive by nature, loves asking questions and some times does get into trouble for doing so. He’s passionate about certain things in life and building solutions that have the ability to impact people’s lives in a positive manner is one of them. He believes that he can change the world and is doing the little he can to change it in his own little ways. When not hacking open source, building new products, writing content for Practical Performance Analyst, dreaming up new concepts or building castles in the air, you can catch-him bird spotting (watching planes fly above his house). You can reach trevor at –  trevor at practical performance analyst dot com. The views expressed on this web site are his own.

Related Posts

  • SPE 101 – Refreshing The Basics – Part IISPE 101 – Refreshing The Basics – Part II This is Part II of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I Introduction - It’s been a while since we have touched upon the basics of SPE (Systems Performance Engineering) here at Practical Performance Analyst. The SPE Fundamentals […]
  • SPE 101 – Refreshing The Basics – Part IIISPE 101 – Refreshing The Basics – Part III This is Part III of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I You can read the second post here - SPE 101 - Refreshing The Basics - Part II Introduction - The intention of this series from the start has been to cover some of […]
  • SPE 101 – Refreshing The Basics – Part IVSPE 101 – Refreshing The Basics – Part IV This is Part IV of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I You can read the second post here - SPE 101 - Refreshing The Basics - Part II You can read the third post here - SPE 101 - Refreshing The Basics - Part […]
  • SPE 101 – Refreshing The Basics – Part VSPE 101 – Refreshing The Basics – Part V This is Part V of a V part series. You can read the first post here - SPE 101 - Refreshing The Basics - Part I You can read the second post here - SPE 101 - Refreshing The Basics - Part II You can read the third post here - SPE 101 - Refreshing The Basics - Part III You […]