Welcome to the Diagnostics Shootout 2013 at Practical Performance Analyst.
There’s been a slight change with regards to selection of tools for the Diagnostics shootout here at Practical Performance Analyst. We’ve decided to include AppNeta Traceview in our Diagnostics Shootout for 2013 in replacement of HP Diagnostics. We at Practical Performance Analyst believe that this is a good outcome for our readers for various reasons.
The new generation of Application Performance Management tools i.e. App Dynamics, New Relic, TraceView are designed to provide the Performance Engineer with a lot of relevant information required to identify potential performance issues. These tools are known to impose some of the lowest management overhead we’ve ever seen at costs that would be the envy of existing APM vendors. The new breed of SaaS based APM tools are revolutionizing the way we collect and analyze Performance data for our applications and as these solutions mature we’ll see a whole lot of interesting capability being delivered at an affordable cost.
In this episode we’ll take a look at AppNeta’s Trace View and what it has to offer to the Practical Performance Analyst. AppNeta’s TraceView is a SaaS based solution for Application Monitoring & Diagnostics. In case you’ve just jumped in and haven’t read any of the previous articles in this series, we highly recommend going through some of the past episodes of the Diagnostics Shootout:
- Introduction to the Diagnostics shootout 2013 – Click Here
- Diagnostics Shootout 2013, App Dynamics – Click Here
- Diagnostics Shootout 2013, New Relic – Click Here
Application Diagnostics as we’ve mentioned earlier is an activity aimed at gaining insight into the behaviour of the application or application container (J2EE or .Net) with the objective of identifying any potential hotspots. One generally uses Application Diagnostics tools to identify application code, application configuration, or infrastructure bottlenecks with the objective of tuning them so as to ensure that Application Business Transaction Response times are within acceptable bounds.
Please Scroll Down to see the Video Recording
What is TraceView –
TraceView is AppNeta’s Application Performance Monitoring & Diagnostics solution offered in a SaaS model. TraceView is currently only available in a SaaS model and has a complementary offering for NPM (Network Performance Management). For purposes of this Diagnostics Shootout we’ll stick to the APM solution. TraceView’s capability can be summarized as follows:
- Ability to monitor application transactions across the stack
- Ability to correlate application business transaction response times to method level response times to response time for the calls to the database
- Ability to perform RUM (Real User Monitoring i.e. capture the performance statistics for real users accessing your application)
- Ability to perform Operating System Monitoring
- Ability to generate emails for alerts and provide reporting
Screenshot – AppNeta TraceView Main Dashboard View
TraceView provides all the features you would expect out of a SaaS based Application Performance Monitoring solution. The Saas based solution offers an easy setup process and absolutely low management overhead which is a perfect fit for DevOps.
What do you get out of the box with TraceView –
TraceView’s capability can be broadly broken down into the following areas:
- Operating System Monitoring
- Application Diagnostics
- Real User Monitoring
The Operating System Monitoring capability offered by TraceView allows you to monitor the following Linux based Operating Systems. Like most other SaaS based APM solutions the support for Operating Systems is pretty much limited. In TraceView’s case it’s limited to 5 mainstream distributions of Linux.
- Linux Mint
- Red Hat
With regards to Application Diagnostics, AppNeta claims that its solution offers ultra-low overheads when used in production environments i.e. <1%. This AppNeta claims is due to the following two key features built into its technology.
- Light weight, non-blocking instrumentation – Performance data gathered from a transactions execution path with the instrumentation adding an overhead of a few milliseconds.
- Sampling – TraceView only performs traces on a small set of samples. This allows TraceView to collect performance statistics for a few key methods by instrumenting them dynamically. The rest of the business transactions are left untouched.
When performing a feature to feature comparison the application diagnostics and profiling capability in TraceView is very similar to what’s being offered by AppDynamics and New Relic.
The application environments supported by TraceView include:
For some reason .Net is missing from the list of supported platforms. This is one area which from a feature standpoint sets TraceView apart from the rest of the competition.
Finally the Real User Monitoring capability in TraceView allows you to capture response times across the following tiers:
- Network response time
- Server response time
- DOM processing time (time spent by the client browser assembling the DOM)
- Page render time
TraceView also provides out of the box reporting and alerting capability. All of the functionality provided by TraceView can be summarized as follows:
- Infrastructure Monitoring –
- Monitoring Linux (RedHat, Ubuntu, CentOS, Linux Mint, Debian)
- Collect out of the box Operating System Performance Metrics
- Application Monitoring & Diagnostics
- J2EE Application Diagnostics with the ability to perform tracing and detailed profiling
- PHP, Ruby and Python monitoring
- Ability to capture application performance and response times across various tiers i.e. from the application server down to the messaging tier and the database
- Business Transaction Monitoring
- Monitor performance of critical business transactions from a Real Users standpoint
- Monitor performance of the business transaction across the client (rendering time), network transfer time, application server processing time and database fetch time
Video – AppNeta TraceView Application Performance Monitoring & Diagnostics
Areas of Improvement –
TraceView with all the capability it offers makes it a strong contender in the SaaS based Application Performance Management space (APM). However there are a few key areas across the tool that are really annoying and need improvement.
- The ability to correlate information across the various tiers left a lot to be desired. We had challenges understanding how to correlate information across the various tiers. This was something that worked out of the box with the other SaaS based APM tools i.e. AppDynamics and New Relic.
- TraceView provides a view of the application transaction as individual application requests which is really annoying. What I would like to see is a “Real Business Transaction” view that provides a view of all the main business transactions and only on clicking them I get details of the individual requests. This is something that’s available out of the box with the other SaaS based Application Performance Management solutions.
- The interface while intuitive to a great extent but has some very annoying features. One of the most annoying features we found was the transaction performance dashboard view that only provided part of the transaction name when listed in a tabular format and you would have to hoover the mouse over each of the transactions or click on it to actually see the whole transaction name. The column with the transaction name was also not resizeable which made things worse.
- The lack of support for .Net was another big gap. Yes, a very large part of the applications out there are Java based but there are still a large number of Microsoft shops.
- Inability to create custom dashboards for both Operating System monitor and application monitors. The Operating System monitoring view requires you to manually select performance metrics every time you pull up the dashboard. This is really annoying especially if you are going to be dealing with a large number of servers and are moving across various screens.
- Inability to zoom in or zoom out and view performance of Operating System metrics. We would recommend larger graphs for the Operating System monitoring data and a more intuitive display of metrics.
TraceView has a lot going for it. It’s definitely a strong contender, has a few rough edges and we are sure over the next few releases AppNeta will work on smoothing them out.
Our Verdict –
List below are ratings for each of the different criteria we had in mind when evaluating the tool. All the ratings are between 1 to 10 with 1 being the lowest score and 10 being the highest:
- Ease of installation and configuration : 9
- Should be light weight with regards to performance overheads i.e. Utilization or Response times post instrumentation shouldn’t go beyond 3-5% : 9
- Dashboards should provide relevant performance data by summarizing the relevant information across the various tiers : 6
- Should provide a Business Transaction view of Performance across the various application components and tiers : 5
- Should have the ability to trace Business Transaction Performance across the various tiers : 5
- Should integrate along with other infrastructure monitoring tools or provide the ability to co-relate application performance with infrastructure performance : 6
- Should be easy and cost effective to maintain: 9
- Should provide the ability to perform detailed profiling required during the test / development phase and also turn level of tracing down to sampling for use in production environments : 6
- Should tie application performance with end user transactional performance and infrastructure performance providing the user a single view of application performance across the various application components and tiers : 6
Closing Notes –
Based on what we’ve seen AppNeta’s TraceView is really easy to setup, configure and get up and running. It took us a whole of 15 minutes from the time of deployment to the time we started generating data for analysis. However, the interface lacks the capability to easily correlate information across the various tiers without too much fancy customization and configuration. This is where AppNeta’s TraceView starts losing out against the competition. We reckon that TraceView will turn out to be strong contender in the SaaS APM space in the longer run. It’s something everyone should have on their bucket list and is definitely a tool we’ll be revisiting in the next round of the Application Diagnostics shootout at the Practical Performance Analyst.
We encourage you to try out TraceView and see how it stacks up against the competition. We encourage you to evaluate the solution at AppNeta. Choose the option at the bottom of the screen and start your evaluation of TraceView.
As always do Send us an email with your input, comments, feedback and suggestions. If you think you’ve got the the talent, are keen on sharing your knowledge / experiences and are keen to help us grow the community here at Practical Performance Analyst please reach out to us Over email.
If you are a vendor, are reading this and are keen to participate in the next round of the Diagnostics Shootout please write to us at trevor at practical performance analyst dot com.