Performance analysis of an XForms framework with the main focus on profiling by example of Chiba


Diploma Thesis, 2007

105 Pages, Grade: 1.0


Excerpt


Content

1. Performance & Profiling
1.1. Performance
1.2. Performance in the Software Development Life Cycle
1.3. Profiling

2. Methods
2.1. Modelling Approach
2.2. Measurement Approach
2.3. Testing
2.4. Benchmarking
2.5. Monitoring
2.6. Exhaustive Analysis
2.7. Hotspot Analysis Approach
2.8. Exploring the Domain
2.9. Verify & Validate
2.10. Tuning Approach
2.11. Conclusion

3. XForms / Chiba Fundamentals
3.1. XForms
3.2. Chiba
3.3. Conclusion

4. Performance Analysis Iteration
4.1. Utilised System
4.2. Profiling Tools Put into Service
4.3. Acquiring Information about Chiba
4.4. Verify Hypothesis About Hotspots
4.5. XForms Test Bundles
4.6. Test Setup
4.7. Verify Hypothesis About Hotspots 3 - XForms Actions

5. Tuning XForms Actions
5.1. Tuning preparations
5.2. Analyse Bottleneck
5.3. Confute Hypothesis
5.4. The Tuning Process
5.5. Prove Enhancement
5.6. Finale

6. Closing

7. Bibliography

8. Appendix 89

9. Indexes 98

Abstract

The goal of this thesis is to analyse a common XForms framework (Chiba) with the focus on profiling for performance inadequacies and to fix them if possible. A prototype of a largely automated performance measurement setup to underpin the analysis and the verification of enhancements has been designed and implemented.

The thesis starts with a theoretical part to specify terms & methodologies. Before the pa-per comes to its practical part the analysed project is introduced. Afterwards a practical performance analysis from the first steps to the narrow of the analysed project is exem-plarily described.

In detail, chapter 1 Performance & Profiling elaborates how to quantify and profile performance in theory. Chapter 2 Methods introduces various methods to analyse performance. XForms and the analysed product Chiba are presented in chapter 3 XForms / Chiba Fundamentals. The implemented performance analysis is described in chapter 4 Performance Analysis Iteration and chapter 5 Tuning XForms Actions.

The Performance Analysis Iteration focuses on the design and implementation of tests to analyse and validate potential performance shortages, while Tuning XForms Actions illustrates how profiling techniques can be utilised not only to analyse but also to enhance a validated shortage. Finally - in chapter 6 Closing- the results of the analysis and the enhancements are reconsidered.

An overview of figures and tables, referenced resources as well as the appendix can be found at the end of the thesis. Project files and measurement results are placed on the at-tached CD.

“You cannot control what you cannot measure” DeMarco, T

1.Performance & Profiling

Before starting the analysis some definitions have to be made as a base for the further thesis.

1.1. Performance

“Someone’s or something’s performance is how successful they are or how well they do something” (cp. [Corb06, p. 1066]).

This definition of performance does not only focus on how someone or something is or does something. But it implies to be or do something successful. Hence, it must be possible to measure performance to analyse someone’s or something’s performance.

To prove that someone is more, equal or less successful a scope within performance can be compared is needed. An example: Stanley won the Grand Challenge (GP) II1. Hence Stanley was performing best within that race. The GP II was a race by the Defense Advanced Research Pro-jects Agency (DARPA) in 2005. Robot cars had to drive autonomously 175 miles through the Mojave Desert (USA). Hence, Stanley was not the driver but a VW Touareg V5 modified by the Stanford University Racing Team. And Figure 1 Bailey's Human within this race in 2005 Stanley was proven to be the best Performance Model performing robot car.

illustration not visible in this excerpt

A model to analyse human performance is Bailey’s human performance model displayed in Figure 1. The three categories and their co-operation can be understood as: somebody (human) does something (activity) in someplace (context). Comparing this to the former example works too: Stanley (machine) was the fastest driving (activity) car and won the Grand Challenge II in 2005 (context).

Before going deeper into the details of performance we take a look why performance matters at all.

1.1.1. Performance Matters

It is almost evident that performance matters not only if thinking of robot cars in a race.

E.g. AOL once pronounced an unlimited online access for just 19,95 US $ a month. Much more people than expected adopted that offer. AOL’s servers were not able to handle the demand since their performance was bad. AOL was accused to act consciously mali-ciously (the offer was mentioned to be beguilement). Although AOL was never convicted it had to pay 320 millions US-$ in various compositions with its customers (cp. [ScSc00, p. 34])

Even higher performance criteria have to be met by so-called real-time systems controlling technical processes. Such a system must imperatively react on external events and proc-ess its tasks within well-defined time limits. If a system does not achieve its performance the whole system usually fails and in the worst-case people might get injured or even killed. This is evident thinking of industrial computers controlling oil refineries or embedded computers in aircrafts.

These examples demonstrate the importance performance has. Not only - like usually seen - as a non-functional requirement but as a functional requirement potentially compromising a whole system (cp. [Petr01]). Also other non-functional requirements like security or usability might become functional requirements in specific contexts too. Obviously an application with the main focus on encryp- tion is expected to define anything concerning security as well as security itself as functional requirement.

The AOL examples illustrates that the absence of performance can cause a damage that might even be measured in money. Not meas-urable in money is the loose of reliance cus-tomers possibly had in AOL. Alike hard to measure is the loss of productivity user experi-ence if an application performs badly and has long idle times (cp. [Hain06, p. 5]).

illustration not visible in this excerpt

All this can lead to a lost of revenue for the company. Figure 2 illustrates how the costs for performance problems arise the later they appear in the software development life cycle.

Following this argument performance should be considered as early as possible. Simply because the later performance problems arise, the more expensive it will be to fix them. How to care about performance within various software development phases will be illustrated briefly. But first:

1.1.2. Software Development Orientated Meaning of Performance

Miles J. Murdocca suggests that the key identifier of machine performance is its execution times (cp. [Mudo99, p. 406]). Next to execution time (also called response-time) the throughput is a common identifier to quantify performance. The throughput is characterised as the amount of transactions per unit of time (cp. [Hain06, p. 26]).

Both terms have a direct relationship: - Response Time: Amount of time needed to accomplish a fix use case
- Throughput: Amount of accomplished use cases during a fix time

Steven Haines [Hain06] esteems performance by the various meanings it has in the Software Development Life Cycle. Different people in different roles all speaking about performance might mean various and quite different things. He points out the application support engineer as for him performance means primarily application response times. The network administrator thinks of the availability of bandwidth if questioned while the database application programmer has only the throughput of a specific database query in mind. The company officer in the end mainly cares how many users can comfortably use its product at the same time if asked about performance.

Consolidating this with a focus on Java 2 Enterprise Edition (J2EE) Haines points out four aspects to quantify performance: Abbildung in dieser Leseprobe nicht enthalten

Apparently other authors try to define performance by quantifying its different aspects too. Steve Wilson and Jeff Kesselman [WiKe01] mention five traits of performance in a common Java environment.

illustration not visible in this excerpt

Scalability is another key word often heard in one breath with performance. Scalability describes the performance behaviour of an application in varying scenarios. Scenarios can be varying machines as well as different loads of users working with the application (cp. [Hain06, p. 224 et seq.]).

The categories chosen by the authors to classify performance seem to be quite different. Wilson and Kesselman might have focused on where performance appears (CPU, RAM). While Murdocca and Haines seem to focus on how performance can be measured (re-sponse - time, throughput, availability). Comparing what is behind these categories it is obvious that they all focus towards the same: To find performance metrics to make performance ascertainable. Key identifiers to measure performance are needed to:

- Compare performance of an application with its former versions or other applications
- Discover application performance if executed within different systems.
- Pronounce minimal requirements to run an application

This paragraph has shown that performance can have multiple meanings even in a specific Java scope. A look where performance and especially performance problems might appear in distributed environment should help to specify how performance is to be understood within this thesis.

1.1.3. Performance Limitations

Any application running on a computer is limited by its surrounding system. It is evident that the application performance is limited by the given hardware configuration:

- Central Processing Unit (clock rate, availability)
- System Memory (size, clock rate)
- Input / Output operations (data rates of local disks or networks).
- Bus system (connecting the other units, bus speed)

But even running on the fastest hardware available, an application might behave slowly. The applied software has as well as the hardware a huge impact on the overall performance of an application. Table 3 outlines exemplarily some possible performance limitations in a typical J2EE environment: Abbildung in dieser Leseprobe nicht enthalten

The names Web Server 1 or Client 1 give already a hint that the reality is even worse. Not only the network is disregarded within the figure but also the possibility of many clients connecting to multiple web servers. Or the application receives its data not from a single but clustered data base system.

One of the most common protocols within various networks is the Transmission Control Protocol / Internet Protocol (TCP / IP). Considered the illustrated scenario takes partly place within the Internet and the communication is based on TCP / IP the next limitations are quickly found. The throughput of TCP / IP depends mainly on two things: bandwidth and TCP window size. Hence a wrong chosen TCP window size can become a performance limitation (cp. [Stev97, p. 354 et seq.]). A single TCP / IP package sent from a client to a web server might pass multiple devices (hops) on its way. These devices could be routers, switches, firewalls or proxy servers. Depending on the device each package is unwrapped, analysed, wrapped again and sent to the next hop. All this devices can cause performance inadequacies.

Also other tasks and applications running on the same operating system as the analysed application can become performance limitations. Any utilisation of system memory or CPU time means less of these resources for the analysed application.

1.1.4. Performance in the Focus of this Thesis

Various metrics to quantify performance have been introduced. The former paragraph gave a first impression why and where performance problems might appear. These simple examples reveal how complex a comprehensive performance analysis might easily be-come.

The analysed application can be executed within umpteen scenarios from standalone on a single computer up to high-end sceneries including clusters or Servlet containers. One of the very few things all scenarios must have in common is that the application to be analysed is running in a Java environment. Therefore the focus of this thesis is the look at the application. Dependencies to other aforementioned system components like hardware, operating systems or Servlet containers will be disregarded.

The focus is to get to know the interior performance behaviour of the analysed application. This could mean to find and analyse bad performing algorithms or memory leaks. Pro-nounced performance metrics while doing this must not be taken absolute. Even regarding them relative can be dangerous if different systems are utilised. A graphical program exe-cuted on a system with 3D graphic card might have quite different relative performance behaviour than it has on a system without 3D graphic chip. Once again, performance has always to be seen in its context.

Based on the mentioned definitions performance is defined as: Performance is how effec-tive a program fulfils its duties and responsibilities. Hence a performance analysis means to glean which component of a program has which impact on the overall performance of the running application. Which aspects of performance are concretely analysed highly de- pends on the performance analysis approach. This will be illustrated within chapter 2 Methods.

1.2. Performance in the Software Development Life Cycle

There are different opinions about whether performance should be paid attention over the whole life cycle or rather in a special phase where the application is analysed and if necessary enhanced.

An argument to implement a specific analysis phase is that developers should rather put their efforts into the development of the application than concerning about performance. This does not mean that they should totally forget everything about performance but they should focus on code writing (cp. [Shir03, p. 396], [Goetz07]). Other authors recommend to explicitly introducing performance within any phase of the software development life cy-cle (cp. [Broe02, p. 421 et seq.]) How this might possibly be done is exemplary displayed in Table 4.

illustration not visible in this excerpt

The analysed application is available in a stable state since July 2005. The upcoming performance analysis takes place within the quality assurance phase.

1.3. Profiling

Profiling is about observing and analysing something or someone (cp. [Corb06, p. 1141]). The challenge is to get to know the characteristics of suspicious subjects to prevent or predict their future behaviour. In software development context profiling does mean exactly the same. There are three common focuses profiling can have in software development:

- Code Profiling
- Memory Profiling
- Coverage Profiling

But before these focuses are explained more in detail some more knowledge about how profiling works technologically is needed.

1.3.1. Profiling Techniques

The two main approaches how profilers are implemented are sampling and instrumenta-tion.

Sampling

The sampling profiler disrupts the running application periodically and collects data about it and its environment. This data are compared to earlier collected data to create statistical evaluations. Disadvantage of this approach is that the evaluation is not very precise and only data provided by the operating system and JVM can be collected. The advantage is, that this technique does hardly affect the running application; it is a little bit slower because the profiler is running too but the profiled application is left unchanged. This makes sam-pling profilers usually faster than instrumenting profilers (cp. [ABLU00, p. 90]).

Typically the following data are collected with sampling profilers:

- CPU load
- System memory and JVM heap usage
- Thread and process id
- I/O operations
- Method invocation, but with the possibility that especially very short methods are missed

Sampling offers a quick first look what happens within the application and its environment during runtime but this look is not very precise.

Instrumentation

Profilers using instrumentation are extending the profiled application with code to gain performance data. Instrumentation in Java can be done in several different ways, for example by the following techniques: Abbildung in dieser Leseprobe nicht enthalten

Instrumenting profilers are much more precise and fine grained then sampling profilers. While sampling profilers usually just show that a method was invoked, instrumenting profilers display the exact amount of times a method was called or inform how much heap space a concrete object allocates.

1.3.2. Code Profiling

Like the name says profiling code focuses on measuring what is going on within the source code. This means to measure execution times and frequencies of method calls or to locate classes and objects allocating a large number of objects or other resources. A code profiler should provide the possibility to filter selected methods and to compute rela-tive execution times for a selected method, what means that the execution time of 100 ms method XY spent during runtime is set to 100% and all methods called deeper in the exe-cution tree are computed newly based on that 100%. This makes it easier to analyse huge applications. A very easy way to profile an application is the hprof profiler included in Java

5. Simply executing an application with java -agentlib:hprof ProfiledApplication starts the internal hprof profiler and it creates a text file called java.hprof.txt to persist its profiling results.

1.3.3. Memory Profiling

Focusing on memory regards everything about physical memory, virtual memory (paging), the Java heap, garbage collection, class and object sizes or the amount of created ob-jects. There are two main reasons for memory driven performance problems (cp. [Hain06,

p. 126 et seq.]):

- Oblivion objects (loitering objects) estimating heap space.
- Frequently created and destroyed objects resulting in copious garbage collection

Reading the results of a profiling session usually cannot easily identify both problems. Considering usual applications with hundreds of objects it is very difficult to decide which object is needed and which not. To detect them the following five steps are recommended (cp. [Hain06, p. 140]):

1. Invoke garbage collection
2. Save heap snapshot
3. Accomplish use case
4. Repeat step 1 and 2 again
5. Compare the two snapshots, identify unforeseeable objects and backtrack their creation.

A good profiler shows how many objects of a class have been created and destroyed between the two snapshots. Easy use cases are recommended here to have the possibility to approximate the maximum amount of objects that should exist.

1.3.4. Coverage Profiling

Coverage profiling means to measure the percentage of code that was executed during runtime. Coverage profiling is not helpful in the beginning of finding a bottleneck, but later on to ensure the quality of created performance tests. For a comprehensive performance analysis the results received from code and memory profiling during performance tests should be treated with suspiciousness as long as coverage profiling shows that only 15% percent of the code expected to be passed, is effected by the tests.

Next to verifying the quality of code by measuring common metrics comprehensive cover-age tools even support the trailing of bottlenecks. Coverage Profilers like Clover [CeClv] do this by offering a graphical overview showing which parts of the codes was executed during runtime by highlighting it. Executed statements are bracketed green and not exe- cuted red. This supports the analyst as it opens the possibility to estimate prima facie which parts of a class were executed. Furthermore, the tools indicate how often an expression was evaluated to true and false and the number of rounds a loop has been executed. Knowing that a conditional statement that was expected to evaluate to false evaluates to true or that parts of methods were not executed although they were supposed to do may be a good indicator to adjust code or memory profiling efforts while searching for the reason of a particular bottleneck.

1.3.5. Profiling Jeopardy

The biggest problem about profilers is the adjustment between the overhead a profiler causes and the amount and significance of measured data. The more detailed a profiler analyses the code, the more impact it usually has on the profiled application since this means that more performance metrics must be measured, so more code is instrumented or more samples are taken.

Further pitfalls concerning the validity of measure data can be found within the paragraph 2.9 Verify & Validate.

2. Methods

"A goal without a method is nonsense. ” W. Edwards Deming

Various static and dynamic, formal mathematical or simulation methods exist to analyse the performance of an application. John and Eeckhout classify these methods into performance modelling and performance measurement (cp. [JoEe06, p. 5]).

2.1. Modelling Approach

Performance modelling is usually practised while an application or parts of it are created from scratch. In this state often tests measuring performance cannot be executed, as essential parts of the system to test do not exist yet. Performance modelling is further divided into the subcategories simulation and analytical modelling.

2.1.1. Simulation Modelling

Simulation Modelling includes techniques like statistical, execution, event-driven and complete system simulations as well as probabilistic models. Like the names suggest they simulate in various ways environments where the application will or may be executed in. But also the application itself or some of its components might be simulated in a simulation modelling context (cp. [JoEe06, p. 8]).

2.1.2. Analytical Modelling

Analytical modelling techniques are formal mathematical approaches like queuing, Mar-kov- or Petri net models. The disadvantage about analytical models is their leak of accu-racy (utilisation within 10% and response times within 30% accuracy) while the advan-tages are low cost and requirements since anyone owning the mathematical abilities can perform an analytical analyse and no additional tools are needed (cp. [JoEe06, p. 15]).

Analytical and Simulation Modelling bear the menace to miss the reality, as they do not operate under the same conditions a real running application does. But John and Eeck-hout demonstrate extensively in their book “Performance Evaluation And Benchmarking” [JoEe06] how these various techniques can be utilised to successfully analyse the performance of becoming applications. Not only if there is no other possibility then using these techniques to predict or analyse the performance behaviour of an application.

Since the application this thesis deals with is already released and even has left the beta status, the focus of this thesis is not on Performance Modelling but on Performance Meas-urement.

2.2. Measurement Approach

While Performance Modelling focuses on the becoming application Performance Measurement focuses on executable prototypes and already built applications. Performance Measurement helps the analyst in understanding the running application on a specific system with possibly different workloads (cp. [JoEe06, p. 16 et. seq.]).

Measured data can be used to

- Validate results gained by performance modelling
- Disclose capabilities and boundaries of the analysed system
- Make statements about the capacities of an application
- Tune the system
- Adjust future versions of the analysed application

Further persisting measured data over a longer period of time provides the possibility of an early warning system. Ideally a continuous integration system periodically measures automatically specified performance metrics and compares them to already existing vali-dated data. If any measured metric exceeds its maximum given by prior valid data the sys-tem generates a report about the specific failure and sends a notification to a given re-source.

Concerning quickly growing measurement results and limited resources versions or branches of the application are good moments to persist measured data. Common techniques to measure performance are tests and benchmarks.

2.3. Testing

Testing does not mean to prove the absence of errors but to demonstrate that the program performs as it is expected to, even in situations that where not expected. This sounds a little bit hair splitting but it is important, as humans tend to follow there once chosen sub-consciously objectives. Thus testers following the objective to verify that an application is free of errors will act subliminal towards this objective. This might happen through bad de-signed test cases or by choosing inaccurate test data. While testers analysing the same application with the objective to find errors will find errors for the same reason the other testers will not find any (cp. [Meye04, p. 6]). In a funny and unscientific but very accurate way Robert Anton Wilson summarises this phenomenon as a common law about the thinker and the prover:

“ What the thinker thinks, the prover proves ” 2 (R.A. Wilson)

Hence testing for performance means to find performance constrains the application does not achieve successfully. Typical constraints are specific amounts and ranges of time and resources within a use case to be accomplished. These constraints should be defined in the SLA’s as exemplary displayed in paragraph 1.2 Software Development Life Cycle.

Test frameworks, like for instance JUnit [JUnit] already inherently measure execution times of executed test cases. These might give a first impression on the performance attitude of the tested application but this quick gained impression is not validated. This impression possibly even deceives the real estate of an application as performance problems might only appear during the interaction of the tested units.

Often performance measurement is done within explicit performance tests, which usually happen during the system testing phase of the testing life cycle. But like the just mentioned example shows other test phases and test techniques can be used to measure performance. The book “The Art of Software Testing” mentions 20 different test techniques (cp. [Meye04, p. 233 et seq.]). Therefore, an elaborated discussion about how specific performance tests might be used or not used to measure performance is impossible within this thesis. Exemplarily some typical test techniques and their relationship to performance can be found in the appendix (q.v. 8.1).

Although, tests without an inherit focus on performance might be utilised for performance measurement the placement of all tests explicitly concerning performance within the system test did not happen without a reason. This categorisation accommodates the fact that meaningful statements about performance are only possible within a defined scope, here the system. Testing techniques with an explicit focus on performance measurement are primarily volume, stress and performance tests.

2.3.1. Volume Testing

Volume tests (load test) focus on the behaviour of the application under its expected load. Load could be the expected amount of users or size of XML files an application has to parse.

2.3.2. Stress Testing

Stress tests are related to volume tests but must not be confused with them. Stress tests as well utilise heavy loads but differently from volume tests they focus on peaks of loads within usually small amount of times (cp. [Meye04, page 134]) to fail a system. A small example illustrates this:

- Volume Test Focus Determine if a xforms insert event can be executed 500 times consecutively
- Stress Test Focus Determine if 50 users in parallel can accomplish a xforms insert event within 350 milliseconds

2.3.3. Performance Testing

Performance tests focus on demonstrating that given performance metrics defined in the SLA’s are not reached by the system. Typical performance relevant SLA’s have been illustrated in paragraph 1.2 Performance in the Software Development Life Cycle. Common performance tests measure response times, throughput and scalability of a system. But performance test can also focus on the performance of the user while interacting with user interfaces (UI).

Typical performance metrics are (cp. [Rubi94, p. 156 et. seq.]):

- Time user needs to find a specific information or fulfil a use case
- Number of times user needed support to accomplish a use case. Ordered by the location providing the support: - Manual / Handbook
- Online help
- Customer call centre
- Number of times user needed back button

These human performance tests could also try to determine the perceived performance of the users while interacting with the UI. A potential outcome of these tests could be to rearrange parts of the UI to better serve the user needs. Or to concentrate on a use case of the application because all users perceived this as slow or needed support.

2.4. Benchmarking

The Standard Performance Evaluation Corporation (SPEC) defines a benchmark as3

“ A test, or set of tests, designed to compare the performance of one computer system against the performance of others ” (SPEC Glossary)

Various benchmark tests exist to compare for instance the performance of various Java configurations and implementations on different computer systems: Abbildung in dieser Leseprobe nicht enthalten

Common benchmarks to evaluate the performance of World Wide Web Servers are the SPECweb2005 benchmark version 1.1 from July 2006 specified by the SPEC. The Transaction Processing Performance Council (TPC) provides benchmarks to measure and compare Application Server and web services4.

Since benchmarks compare performance on different computer systems considering meaningful benchmark data is a very comprehensive task and usually only applicable for basic technologies (cp. [Petr01]). Because XForms is not a technology but a specification is hard to consider valid benchmarks targeting on various XForms implementation. This means to compare and consider not only one basic technology but also various independ-ent technologies as XForms might be implemented in multiple computer languages. Even creating a benchmark for a single XForms implementation will be hard work as this neither is a basic technology nor utilises basic technologies like a simple example quickly demon-strates:

The XForms implementation Chiba possibly runs on XY devices from the Java enabled cell phone to clustered application servers. Further it utilises YZ technologies. As there are Java, XML and its derivatives, JavaScript and CSS. Even if only browsers are used for rendering the possible combinations are getting fast very complex. A benchmark would have to consider all these various facts regarding performance of XY processors to YZ compilers, interpreters and what ever is needed. It is obvious that this is not impossible but probably not worth the trouble.

In contrast various authors use benchmarks not to compare the performance of one sys-tem to another but to measure performance within one system. In this meaning a bench-mark just measures performance metrics within a defined and specified system. These can be compared to other results measured on identical systems. Interestingly none of the books using the term benchmark in this way provided a definition for it (cp. [Shir03, p.12]). E.g. two Java XForms implementations could be compared if they run under almost equal conditions. The more different underlying techniques are used, the harder it will be to ad-dress performance differences to the implementation or its enfolded system.

2.5. Monitoring

Monitoring is a further approach for Performance Measurement. Monitoring means to ob-serve the running system during runtime fulfilling its destination. Ideally this happens in a real life situation with an approach of unattended monitoring 24 hours a day, 7 days a week measuring various performance metrics. But monitoring need not be that extensive as it can also mean to browse through server and application log files to gather performance relevant data. As monitoring has lots of parallels to profiling it has the same impact on the monitored system as profilers have: it slows down the system. This is not happening if log files are generated anyway and these are analysed later, but usually monitoring goes further than this as the following paragraph demonstrates.

Most modern Java monitoring tools utilise the J2EE Management Specification to monitor performance metrics of the observed system but also Applications Servers take advantage of monitoring techniques like the Java Management extension (JMS).

2.5.1. J2EE Management Specification (JSR-77)

In July 2002 the Java Community Process (JCP) released the J2EE Management specification - also called Java Specification Request 77 JSR-77) in its final version. The last maintenance release is from June 2006 [JSR-77].

The domain of the specification request is to provide functionalities to:

- Detect and drive managed objects within a managed system
- Obtain advice notes about important events dispatched on managed objects
- Monitor and manage the running state of managed objects
- Observe basic performance metrics of managed objects

illustration not visible in this excerpt

JSR-77 enables fast development of management solutions and offers the possibility to manage multiple implementations of the Java platform within a single management tool. Concerning at least Java the JSR-77 is the most comprehensive vendor neutral specification concerning performance management (cp. [Hain06, p. 50]).

As shown in Figure 4 a huge part of the JSR-77 (77.6) focuses on Performance Management. JSR-77.6 provides a performance data frame-work enfolding a statistic provider model (StatisticProvider) and statistic interfaces to collect meas-urement data. The StatisticPro- vider defines requirements for performance monitoring. Each J2EE Managed Object must implement its provider interface. There is a Serv- letStats interface for Servlets, an EJBStats interface for EJBs and so on. On the other side of the track Statistic interfaces are representing common performance metrics (cp. Figure 5).

illustration not visible in this excerpt

Any returned data by the interfaces is from the native type long. Typical methods within the interfaces are for instance getCount(), getMaxTime(), getTotalTime() defined in the TimeStatistic() interface or for instance getUpperBound() and getLowerBoundor() defined in BoundaryStatistics().

The most common approach to gather any information provided by the JSR-77 to utilise them in monitoring tools like Chainsaw or profiling tools like YourKit Java Profiler is the Java Management extension (JMX) specification.

2.5.2. Java Management Extension (JMX)

The original Java Management extension was specified 1998 originally as JSR-3. New aspects of JMX are specified in JSR160 JMX Remote API5 and the JMX 2.06 is already in progress called JSR255. Current JXM version is 1.4. The J2EE Management (JSR-77) specification is built upon the JMX API. To provide these possibilities JMX defines next to architecture and API also services to manage and monitor applications within the Java programming language (cp. [JSR-03]).

To fulfil this purpose the interior JMX architecture is divided into the three levels:

- Agent Level,
- Instrumentation Level and Additional
- Additional Management Protocol APIs.

Figure 6 shows the relationship between the agent and instrumentation level as well as the relationship to a distributed service level. Distributed services levels are marked light grey, as they are not yet part of the specification.

illustration not visible in this excerpt

Figure 6 displays a typical three-tier model. The lowest tier, the instrumentation level speci-fies manageable resources like applications, devices or a component defined by a devel-oper. These managed resources are instrumented by at least one but possibly more Man- aged Beans (MBean). Any JMX compliant agent in the second tier, the agent level, can automatically manage and directly control an MBean. An MBean Server ensures that JMX Management applications from any vendor can access registered MBeans through the agents without the need to know a specific agent. Any application in the distribution level can utilise provided protocol adapters and connectors (cp. [JSR-03, p. 24 et seq.]). The JMXConsole7 provided by JBoss Enterprise Application Platform is a well-known example concerning JMX.

The profiling tool YourKit Java profiler obviously participates in the possibilities offered by JMX and J2EE Manage- ment although there is no official proof, as they do not mention any technolo-gies how the product is im-plemented. But error mes- sages like the displayed one might be not a proof but at least a large indication for this assertion (cp. Figure 7).8

illustration not visible in this excerpt

Various methodologies how to analyse and measure performance are emphasised at this point of the thesis. But a structured approach applying all these knowledge is still missing.

2.6. Exhaustive Analysis

An exhaustive performance analysis means the systematic and comprehensive examina-tion of the performance behaviour of a specific application and optionally its environment too. Considering the difference between application specific and reusable code, Jack Shi-raz remarks, that developers of application specific code usually know what their users will do with the developed application. Hence they can concentrate on analysing the performance behaviour of the corresponding use cases. Reusable code brings the chal-lenge with it, that it may be utilised in a way no developer has ever foreseen and hence has not analysed.

This difference is important because knowing what users intend to do with the application makes it possible to analyse the performance behaviour quite comprehensive. In contrast Shiraz refers to class libraries, frameworks, components and beans as explicit reusable code and points out:

“ Without knowing how the code will be used, it is very difficult to create tests that appropriately determine the performance of reusable code. There is no truly valid technique that can be applied. ” (cp. [Shir03, p. 397])

In his originally 1979 written book “The Art of Software Testing” [Meye04] Glenford J. Mey-ers points out nearly the same problem considering the economics of testing. By the ex-ample of input tests and the necessity to run them against an infinitive amount of possible data he comes to the conclusion that exhaustive input testing is impossible. This impli-cates the impossibility of error free code and the assumption that the level of comprehen-sive testing is an economical question (cp. [Meye04, p. 10]). Economical because in the following chapters he mentions various techniques like for instance the Black Box Testing approaches, Boundary Value Analysis and Cause Effect Graphing as instruments to guar-antee nevertheless high-quality error tested code (cp. [Meye04, p. 44]).

Shiraz explicitly does not negate the possibility of an exhaustive performance analysis but mentions he does not know any. A few lines later he points to the same argument Meyers uses regarding exhaustive testing the costs. Due to the limited amount of time and fi-nances this argument applies to this thesis and prevents an exhaustive performance analysis of Chiba during the accomplishment. Instead of a broad performance analysis where as much functionality as possible are analysed concerning their performance be-haviour the focus of this thesis is an exemplary but deeper analysis of some tailored func-tionalities of Chiba.

Another approach to analyse the performance of an application is the Hotspot Analysis Approach.

2.7. Hotspot Analysis Approach

An article about a unified performance analysis approach published in the IBM System Journal Volume 39 No 1 [ABLU00] describes the common approach of a performance analyst as follows:

“ Measure performance, find constraints to the level of performance achieved, eliminate or reduce their effects, and then start again; stop when measured performance achieves a previously agreed-to target (cp. [ABLU00, p. 118])

This approach bases on a common rule, the Pareto rule, which says that most impact on something comes from few activities, while most activities have only few impact on that something (cp. [Gupt03, p. 31]).

Translated to a performance analysis of an application this means that the application spends most of its execution time in only a small part of the code, the hotspot, while the majority of the code does not affect the execution time heavily. A common metric here is the 80/20 rule, where 20% of the source code affect 80% of the application execution time (cp. [Shir03, p. 397]). It is not important how exactly this proportion is; it might only be 60/ 40 in a small balanced program or it goes up to 90/10. Undertaking this rule an iterating performance analyse approach including the fixing of found hotspots like mentioned by IBM is a very efficient and passable approach to receive meaningful performance metrics about an application without spending time analysing irrelevant parts of the program.

The following hotspot approach is implemented within this thesis:

1. Acquiring information about the domain, appoint hotspots
2. Verify found hotspot(s)
3. Define measurable performance requirements ought to be achieved
4. Analyse thoroughly the hotspot and identify underlying bottlenecks
5. Fix one bottleneck by the time considering the tuning approach until performance requirements are reached (q.v. 2.10 Tuning Approach)
6. Start again with step one

Hotspots are any points of interest concerning any of the mentioned performance quantifi-ers. A single hotspot is usually caused by one or more bottlenecks and must not be con-fused to be the bottleneck itself like displayed in Table 8. An existing bottleneck might in-fluence other bottlenecks since they might be hidden by the actually examined bottleneck or altering one bottleneck might fix another one too. For this reason only one hotspot and within this only one bottleneck should be fixed by the time (cp. [Shir03, p. 18])

illustration not visible in this excerpt

Table 8 demonstrates how important this separation of hotspot and bottleneck can be. The regarded possibly XPath bottleneck is placed outside the Java code within the XForms source code. An analyst focusing only on the measured execution times might start to tune the database controller, as he does not even notice the real bottleneck. Paragraph 2.9 Verify & Validate shows more detailed how measured results and announced hypothesis about underlying bottlenecks have to be verified and validated to avoid this. The paragraph 2.10 Tuning Approach provides a formal approach to implement this critical retrospection.

[...]


1 DARPA. Grand Challenge. 2007. http://www.darpa.mil/grandchallenge05/gcorg/index.html 1

2 Wilson, R.A.: Prometheus Rising. New Falcon Publication, 2000. 12th printing. ISBN-1-56184-056-4

3 SPEC. Glossary. 2006. http://www.spec.org/spec/glossary

4 TCP. TCP-App. 2004. http://www.tpc.org/tpc_app/default.asp

5 JCP. JMX Remote API. 2007. http://jcp.org/en/jsr/detail?/id=160

6 JCP. JMX 2.0. 2007. http://jcp.org/en/jsr/detail?id=255 19

7 JBoss. JMXConsole. 2007. http://wiki.jboss.org/wiki/Wiki.jsp?page=JMXConsole

8 YourKit. Forum. 2007. http://forums.yourkit.com/viewtopic.php?p=4492& sid=97614bc20b04ae7597a048c5a41a05c9

Excerpt out of 105 pages

Details

Title
Performance analysis of an XForms framework with the main focus on profiling by example of Chiba
Grade
1.0
Author
Year
2007
Pages
105
Catalog Number
V85134
ISBN (eBook)
9783638900164
ISBN (Book)
9783638905657
File size
3482 KB
Language
English
Notes
Die Arbeit ist von Prof. Dr. Roland Petrasch für den Erhard Höpfner Studienpreis eingereicht worden. Der Preis wird im Februar 2008 vergeben.
Keywords
Performance, XForms, Chiba
Quote paper
Diplom Informatiker Lars Windauer (Author), 2007, Performance analysis of an XForms framework with the main focus on profiling by example of Chiba, Munich, GRIN Verlag, https://www.grin.com/document/85134

Comments

  • No comments yet.
Look inside the ebook
Title: Performance analysis of an XForms framework with the main focus on profiling by example of Chiba



Upload papers

Your term paper / thesis:

- Publication as eBook and book
- High royalties for the sales
- Completely free - with ISBN
- It only takes five minutes
- Every paper finds readers

Publish now - it's free