The Value and ROI Of Performance Modelling

What Is Performance Modelling – Performance Modelling is the science of using analytical, statistical or simulation modelling techniques to build models that are able to simulate the behavior of the system for purposes of forecasting system performance due to changes in user or business workload. Modelling techniques have been used for purposes of forecasting across business for decades, stock market analysts have been using modelling techniques to provide their subscribers a view on where a particular stock or market might move and industry analysts use past historical data to forecast potential future trends. However, modelling isn’t always about relying on data from the past, but rather about taking the learning and insight gained from the past to predict what the future might look like. Modelling or forecasting is a tough game and the more experience you get, the quicker and sooner you make mistakes and learn from the them the sharper your projections are going to be. You won’t always be right and you shouldn’t always be expecting to be right with your forecasts. Performance Modelling techniques are a great opportunity for Performance Engineers to provide a view into the future based on learning and insights gained from experiences of the past using industry standard modelling techniques.

The ROI of Performance Modelling – Let’s not kid ourselves – performance modelling is a complex undertaking. Complexity can be said to be directly proportional to the number of transaction classes which required to be modelled, the number of components in the model, and to the detail rendered in the model itself. Constructing a performance model can also be perilously manual, introducing risks of errors at every step – validation is a painstaking process. Creating a performance model requires patience and diligence!

Measuring the value of performance modelling – So – given the effort – what is the reward and benefit of performance modelling? A good performance model can find potential application bottlenecks, make tuning recommendations, right-size infrastructure plans, and also provide a scalability plan for when the application is rolled out – something that can assist service capacity management. For mission-critical applications, in a waterfall release process, this diligent engineering process mirrors those processes often found in civil or military engineering projects. The question for performance modelling is whether it can be as relevant for agile development processes. The nature of the speedy release cycle means that frequent application profile changes can radically undermine the accuracy and fidelity of the performance model.

Creating a performance model – It’s imperative with any modelling process to consider both the question that you’re attempting to answer, and the data that’s available. Witness the failed analysis with insufficient data, or the accuracy lost because of weak data. Start by creating a performance profile of the application. If we were to focus on network profiling alone, then network trace would be a good tool. However that won’t be much help when profiling server or storage activity. However the more tools we use, the greater the challenge in melding the profiles together.  For this reason, I advocate starting with a strong APM tool – although this will constrain the outcomes.

From this point, we need to choose a good modelling platform. Whilst this could be achieved with any industry standard modelling tools, possible even Excel – I would advocate using a tool designed for this purpose. Of course, this would have to be able to accommodate APM type of profiles. The tool would have to be quick and painless to use – in order to keep the manual effort low, and give us the low cost of ownership we need for good ROI.

Horses for courses – Depending on the nature of your forecasting requirement you would need to choose the relevant modelling technique. Performance modelling techniques ideally leverage one of the following approaches –

  • Analytical modelling
  • Simulation modelling
  • Statistical modelling

Your choice of a modelling technique is highly dependent on what you are trying to achieve, what’s type of data/variables you current have available at your disposa, the nature of time/resources you have at your disposal and finally the amount of effort you are willing to expend to achieve a particular level of accuracy i.e. trade off between accuracy v/s obtaining a sense of direction.

Conclusion – Performance modelling can give us a really strong ROI for those mission critical applications that have a high-cost of failure. It can help accelerate release cycles by cutting out costly and lengthy testing practice. It can reduce performance defects on release, by predicting those bottlenecks that testing would not have found. However, the costs can be high – and must be mitigated by effective tooling, or the practice will perennially be captured and lost through individuals. Assess clearly whether the modelling or the profiling exercise delivers the most value – it is possible that a good monitoring and profiling tool gives clearer course of action. Should you elect to consider the ROI of performance modelling, Excel may be a clear option for a one-off exercise but is likely to present a higher cost of ownership in the longer-run.

For readers who are interested in learning more about Performance Modelling please visit the SPE Fundamentals section at – Link.

Robert Limbrey (LinkedIn) is an experienced SPE professional based out of Nottingham in the UK and currently is pRobert_Limbreyart of CA as the EMEA Products Lead. As part of his role at CA Robert is responsible for driving Capacity Management and Mobility solutions into CA’s EMEA business.  His duties include defining the Go-to-market strategy, Creating synergies through partnerships, Sales Mentoring, Driving diligent consultative process through presales including Sales Analytics. Robert is a high Impact Sales and Consulting Executive, taking technology to markets via C-level propositions and a diligent consultative approach

Related Posts

  • Modelling System Performance Using Fundamental Laws – Littles LawModelling System Performance Using Fundamental Laws – Littles Law In the first article of this series we reviewed some of the most basic Performance Modelling techniques using fundamental performance equations. The first part of this series covered a Performance Modelling technique we call "Response Time Theory" and leveraged easy to use performance […]
  • Application Of Performance Modeling Techniques Application Of Performance Modeling Techniques What is Performance Modeling – Performance Modeling is the process of modeling application performance considering various growth factors with the objective of proactively predicting potential breach of SLA’s. Performance Modeling is also used to validate design decisions and […]
  • Use of Quantitative Models For Performance Testing & Engineering – Part IIUse of Quantitative Models For Performance Testing & Engineering – Part II This article is the second in a two part series by Author Ramesh Iyer on the 101 around Use of Quantitative Models for Performance Testing & Engineering. If you haven't read the first part we encourage you to check it out at the following link - Use of Quantitative Models for […]
  • Solving Enterprise Applications Performance Puzzles: Queuing Models to the RescueSolving Enterprise Applications Performance Puzzles: Queuing Models to the Rescue Introduction  - "Solving Enterprise Applications Performance Puzzles: Queuing Models to the Rescue" is a unique book that complements existing literature on performance modeling. The author of this book Leonid Grinshpan (who also happens to be an author at Practical Performance Analyst) […]