Building Systems That Perform : Workload Characterization – Doing It Right

Dr. Leonid Grinshpan (LinkedIn) is currently Technical Director at Oracle with a focus on Enterprise applications capacity planning, load testing, modelling, performance analysis, and tuning. Leonid has a few decades of experience on two complementing areas: computer science and information technology (IT) engineering. He holds a Ph.D in computer science is is also an author of a book on mathematical modelling.

In this series of articles Dr. Leonid Grinshpan (LinkedIn) presents an approach that Performance Architects or Performance Engineers should consider when attacking Performance Engineering related activities across the SDLC. Dr. Leonid Grinshpan (LinkedIn) has decades of experience building and delivering applications that perform. Through this series he sums up some of the learning that might prevent you from making the same mistakes when addressing Systems Performance across the SDLC.


You can read the first post in this series by clicking here – Building Systems That Perform : Part 1

You can read the second post in this series by clicking here – Building Systems That Performance : Part 2

Introduction – Enterprise Applications are designed to process workload generated by user actions against a documented set of Non Functional Requirements and Service Level Agreements (SLA’s). To ensure that the Enterprise Application being designed, built and delivered is capable of managing the production workload, the Performance Engineering team is expected to carry our relevant design optimization, code tuning, configuration tuning, performance testing, application diagnostics and capacity planning activities across the Systems Development Life Cycle. Towards that end ensuring correct specification of production workload is a key to ensuring that the application being delivered will meet business and customers expectations. Get the workload for the Enterprise Application wrong and you’ve got a major challenge on hand. Your Capacity Plans, Non Functional Requirements, Performance Testing Results and tuning across the stack might be in complete vain.

This post analyses the importance of Workload Modelling and demonstrates that ignoring or skewing production workload specifications negatively impacts the validity of capacity planning and performance tuning recommendations.

Application users communicate with the Enterprise Application by issuing the transactions that represent the requests to perform particular application functions. Each customer or application request triggers a transaction within the system with an indivudual execution process that consumes various hardware and software resources across the stack i.e. Compute Resource, Memory Resource, Network Resource, Software threads, Database connections, etc. Some examples of transactions are:

  • Login into system with user name and password
  • Calculate revenue of plasma TV sales in New England stores in August 2012
  • Estimate the expenses to launch new product line in China
  • Create total company profit and loss statement.

Each transaction ends up when the user interaction completes, application has performed the processing within the system and delivered the relevant responses to the end users. As an example, Figure 1 shows a financial report generated as a reply to a reporting transaction [1].

image_1_11104

Figure1 Financial report generated as a reply to a reporting transaction

Workload Modelling – Rich functionality supported by any Enterprise Application for each line of business is reflected in a broad assortment of transactions available to the end application users. Some Enterprise Applications have hundreds transactions accessible through several different types of user interfaces. Such a richness of Enterprise Application front-ends sometimes causes conflicts with the need for simplicity and performance from an End User Experience standpoint. Nevertheless, Enterprise Applications are designed to support relevant business process and must to a great extent mirror the complexity of the business process while keeping the application stack underneath as simple, scalable and high performing as possible.

The flow of business transactions through a system constitutes application’s transactional workload. The intensity of the transactions at any given time depends on a number of users actively interacting with a system. It also is a derivative of the think time between any two consecutive transactions i.e. Think Time is the Time between two consecutive customer action. The interval between consecutive transactions from the same user can be substantial as a user needs time to assess a reply from an application to a previous request, and to prepare the next one.

A workload model or workload specification for any application  includes the following :

  • A list of transactions
  • A number of concurrent users initiating each transaction
  • And per each transaction its rate measured in a number of times it is set off during one hour by one user
  • Think time between two actions
  • Assumed Response Time between the two transactions

This is an example of characterization of a users-generated workload

image_2_11104

The table indicates that :

  • Business users request every hour 100*5 = 500 financial reports
  • Launch consolidation of sales data 25*2 = 50 times
  • And enter a quantity of sold on-line products 400*2 = 800 times

Let’s assume that the table specifies a real production workload and the Enterprise Application being designed is appropriately sized and tuned for this workload based on documented Non Functional Requirements and associated service level agreements.

What If – Let’s assume that we are using workload models for a given Enterprise Application for purposes of load testing. What if the Workload Models we’ve used for purposes of Performance Testing for the given Enterprise Application generates a workload with downward or upward deviations from the numerical values in the table?

In this case our load test does not generate a realistic production workload, and under such a skewed workload the Enterprise Application will have demonstrate performance which will deviate greatly from the actual production environment. You will also end up with infrastructure capacity which is either low or insufficient or possibly which is far higher than what is initially required. Tuning the Enterprise Application for a skewed workload is technically doable, but because such workload is ficitional , our tuning efforts are not just useless, they are misleading, they degrade quality of service and might lead to unnecessary expenses if we recommend to purchase additional hardware.

While executing performance tuning and capacity planning projects, we have to be well aware of how representative are our manufactured workloads of real production workloads.  Let’s consider presented on Figure 2 results of a load test conducted using LoadRunner, and find out how a workload, generated by this test, associates to a production workload.

image_3_11104

The test includes the following transactions executed in such an order:

  1. User logs in
  2. Open Planning application
  3. Open Web Form
  4. Save Form
  5. If test time not expired, go to 1, otherwise stop the test

As column “Pass” indicates, each transaction was executed 372 times during test which lasted 7 minute and 37 seconds. That is equal to 372 / 7 min 37 sec = ~ 50 executions of each transaction every minute. Because the number of concurrent users is 36, every user initiated one transaction (50/36)*4 = ~5.6 times per minute. Definitely, a workload generated by this test is not production but stressful. This workload can be used for testing stability of the system servers, but not for sizing and tuning of production environment.

Detailed discussion of EA transactional workloads and importance of their specifications for proper EA sizing and tuning can be found in a Chapter 3 of the book [2].


Dr. Leonid Grinshpan (LinkedIn) is currently Technical Director at Oracle with a focus on Enterprise applications capacity planning, load testing, modelling, performance analysis, and tuning. Leonid has a few decades of experience on two complemenLeonid_Grinshpan_v0.1ting areas: computer science and information technology (IT) engineering. He holds a Ph.D in computer science is is also an author of a book on mathematical modelling. Leonid has worked on over 200 capacity planning and performance tuning projects for Fortune 500 customers over the past eleven years. Leonid is also the recipient of the highest award in the USSR for excellence in IT engineering – Award of the Counsel of Ministers of the USSR for design and implementation of an open CAD system for microprocessor based products. He tested out his entrepreneurial skills by co-founding the first Belarusian-American Joint Venture Software Security Belarus (acquired in 1997 by USA company Rainbow Technologies).

Solving Enterprise Applications Performance Puzzles: Queuing Models to the Rescue

Price:

4.6 out of 5 stars (3 customer reviews)

0 used & new available from

Related Posts

  • Building Systems That Perform : Looking For A Needle In A HaystackBuilding Systems That Perform : Looking For A Needle In A Haystack Dr. Leonid Grinshpan (LinkedIn) is currently Technical Director at Oracle with a focus on Enterprise applications capacity planning, load testing, modelling, performance analysis, and tuning. Leonid has a few decades of experience on two complementing areas: computer science and […]
  • Building Systems That Perform : Application Awareness Building Systems That Perform : Application Awareness Dr. Leonid Grinshpan (LinkedIn) is currently Technical Director at Oracle with a focus on Enterprise applications capacity planning, load testing, modelling, performance analysis, and tuning. Leonid has a few decades of experience on two complementing areas: computer science and […]
  • What Do I Do, My System Is Running Like A Dog?What Do I Do, My System Is Running Like A Dog? A long time ago, I presented on a topic "Measuring Response Times of Code on Oracle Systems." The presentation was at a CMG (www.cmg.org) conference and paper for this presentation was a subset of "For Developers: Making Friends with the Oracle Database." In the presentation, I spent a […]
  • A Step by Step Guide to High Performing Websites and Web Applications "Optimizing Digital Experience - A Step by Step Guide to High Performing Websites and Applications " is your practical learning guide for how to deliver great digital experiences to your end users. Get the essential information you need to reduce user wait times for better audience […]