In this thesis we propose another approach targeted at web applications that use Ajax, HTML5 and other modern web technologies to achieve a look and feel that is only known by desktop applications. These web applications are also referred to as Rich Internet Application (RIA).
The user interface is a part of an application the same way its business logic is. This requires software engineers to test the user interface to verify its correctness to the same extend they test the rest of the application.But testing graphical user interfaces is a tedious task without standardized testing technologies engi-neers use for business logic verification like unit testing.To address this shortcoming, research groups applied the concept of model-based testing to user interface testing. In model-based testing an abstraction of the actual graphical user interface is used and test sequences are generated from the model. The model as a formal specification can also be checked by a model checker or pose as a test oracle to validate the correctness of the application.Ideally such a model, or specification, can be generated, or mined, by an automated process. This process is usually referred to as “specification mining” and relieves the test engineer of the task to keep the specification up to date. Many tools and approaches have been proposed to target different kinds of applications and properties that the mining tool infers.
At europe’s biggest software company SAP, where this thesis was manufactured, many RIAsarebe-ing developed based on a user interface library called SAPUI5. This library helps developers to create uniform looking RIAs. To test their graphical user interface and ensure quality software, our approach supports this testing process by using a web crawler known from search engines to automatically infer a specification or model from these interfaces. We achieved this, by modifying the Ajax-capable crawler Crawljax by A. Mesbah and using a new, more efficient and modular state abstraction based on XPath expressions. Our CrawljaxXT also addresses many bugs and shortcomings of the original Crawljax and thus allows for reliable creation of complete user interface models without overly detailed state abstractions.Concluding we successfully evaluated our mined models and its testing capabilities with a modified version of the free model-based testing tool Graphwalker.
Contents
1 Introduction
1.1 Motivation
1.2 Contribution
1.3 Outline
2 Related Work
2.1 Specification Mining
2.1.1 Inferring Formal Specifications
2.1.2 Inferred Properties
2.1.3 Representations of Formal Specifications
2.2 Model-based Testing (MBT)
2.2.1 Graphwalker
2.2.2 ModelJUnit
2.2.3 GUITAR
2.3 Graphical User Interface (GUI) Testing
2.3.1 Test Case Generation
2.3.2 GUI Coverage Criteria
2.4 Web Application Testing
2.4.1 Selenium
2.4.2 Crawljax
2.4.3 WebGUITAR
2.5 Web Technologies
2.5.1 Asynchronous JavaScript and XML (AJAX)
2.5.2 XML Path Language (XPath)
3 Specification Mining on the User Interface Level
3.1 Teminology
3.2 Inferring a Model
3.2.1 Using the Full Document Object Model (DOM) - The Crawljax Approach
3.2.2 Full DOM Approach - Second Iteration
3.2.3 HTML of Interaction Elements Approach
3.2.4 XPath of user-defined Elements Approach
3.2.5 General Approach
3.3 Final Model
3.4 CrawljaxXT: Design and Implementation
3.4.1 XpathStateBuilder
3.4.2 Multiple Click Elements
3.4.3 Plugins
3.5 GraphwalkerXT: Design and Implementation
3.5.1 Testing Infrastructure
3.5.2 State Oracles
3.5.3 Path Generators
4 Evaluation
4.1 Evaluation of other MBT Tools
4.1.1 Crawljax
4.1.2 (Web)GUITAR
4.2 Evaluation Setups
4.3 Quantitative Comparison to Crawljax and Additional Implementations
4.3.1 Comparative Setting “Vanilla”
4.3.2 Comparative Setting “Comparators”
4.3.3 Comparative Setting “ClickOnce”
4.3.4 Evaluation Results of Quantitative Comparison
4.3.5 Additional Implementation Setting “FuzzyMatch2”
4.3.6 Additional Implementation Setting “MultipleClick2”
4.3.7 Additional Implementation Setting “MBTPlugins2”
4.3.8 Additional Implementation Setting “RandInput2”
4.3.9 Additional Implementation Setting “ResetDB2”
4.3.10 Quantitative Evaluation Results of Additional Implementations
4.4 Qualitative Evaluation of Testing Capabilities
4.4.1 Capability Setup “SeedetEdgeErrors”
4.4.2 Capability Setup “SeedetStateErrors”
4.4.3 Capability Setup “RealErrorDetection”
4.4.4 Evaluation Results of MBT Capabilities
5 Conclusion & Future Work
Research Objectives & Topics
This thesis aims to develop an automated approach for "specification mining" on the user interface (UI) level, specifically targeting modern web applications (Rich Internet Applications). The primary research goal is to overcome the limitations of existing testing tools, such as the state space explosion problem and the inability to handle dynamic UI changes, by creating an automated model inference technique that supports effective model-based testing (MBT).
- Automated specification mining for Rich Internet Applications (RIAs)
- Enhancement of the Crawljax crawler with XPath-based state abstractions
- Handling dynamic UI elements, such as overlays, to ensure reliable model generation
- Development of a custom testing infrastructure integrated with Graphwalker for model-based testing
- Evaluation of the approach against existing tools and empirical validation of fault detection capabilities
Excerpt from the Book
Fuzzy Matching
In our XpathStateBuilder we also implemented a fuzzy matching algorithm, that introduces a certain fuzziness into our state representations and removes another kind of false positives from our MBT results. The reason for these false positives is the same that we already introduced in our first approach where we used the full DOM to find a state representation.
CrawljaxXT created two different states when opening and closing an overlay: One before and one after the overlay was visible. This happened even though we still had the comparator running that removed invisible elements.
The problem was a bit more complex because it only appeared when we used the model for MBT. The general rule we inferred from it was that if we remove elements from the DOM during mining, we also have to remove them while generating and walking test sequences on the graph. Else the results have to be contradictory. To determine which elements to remove by only looking at the current DOM during testing is very hard to determine. To still be able to conduct correct MBT we decided not to change any structure of the DOM by removing elements from it. Instead we changed all comparators to “empty” instead of “remove” elements that are invisible or not displayed as explained in section 3.2.4.
Summary of Chapters
1 Introduction: Provides an overview of the evolution of user interfaces and outlines the motivation for automated testing of modern Rich Internet Applications.
2 Related Work: Reviews existing literature on specification mining, model-based testing, and web technologies relevant to UI testing.
3 Specification Mining on the User Interface Level: Details the design and implementation of CrawljaxXT, introducing modular state abstractions, XPath-based identification, and plugins to handle dynamic web application behavior.
4 Evaluation: Presents a quantitative and qualitative assessment of the approach, comparing it against original Crawljax and validating its error-detection capabilities through experimental setups.
5 Conclusion & Future Work: Summarizes the contributions of the thesis and discusses potential future improvements, such as alternative state builder implementations.
Keywords
Specification Mining, Model-based Testing, Web Application Testing, UI Testing, Crawljax, Graphwalker, XPath, Rich Internet Application, RIA, AJAX, DOM, State Abstraction, Test Automation, Software Quality, Regression Testing
Frequently Asked Questions
What is the core focus of this research?
The thesis focuses on automating the creation of software specifications from the user interface of modern web applications to enable reliable and efficient model-based testing.
What are the primary thematic fields covered?
The research intersects software engineering, automated testing, formal methods (specification mining), and modern web technologies (AJAX/HTML5).
What is the main objective of the proposed solution?
The goal is to relieve test engineers from manually creating and maintaining UI specifications by automatically mining models that are robust against dynamic web interface changes.
Which scientific methodology is employed?
The author uses iterative empirical software engineering research, developing extensions for the existing tool Crawljax and evaluating them through quantitative comparisons and fault-seeding experiments.
What is the scope of the main part of the thesis?
The main part covers the design and implementation of "CrawljaxXT," the development of new state comparison comparators, and the integration of these models into the Graphwalker testing framework.
Which keywords best characterize the work?
Key terms include Specification Mining, Model-based Testing (MBT), Rich Internet Applications (RIA), XPath, Crawljax, and Test Automation.
How does this approach handle the "state explosion" problem in UI testing?
By implementing a specific state abstraction using user-defined XPath expressions, the approach reduces the complexity of UI states, ensuring they are uniquely and precisely identified without being cluttered by irrelevant dynamic details.
Why are standard crawlers insufficient for testing these web applications?
Standard crawlers often fail to recognize identical states after UI interactions (e.g., closing overlays) due to underlying DOM changes, leading to the creation of "false positive" states and preventing a complete navigational model.
How does the "OverlayAwareShortestPath" generator improve testing?
It ensures that test sequences generated for web applications with layered UI components (overlays) are logically valid by preventing illegal navigation paths, thereby significantly reducing false positives during model-based testing.
- Quote paper
- Fabian Richter (Author), 2012, Specification Mining on the User Interface Level, Munich, GRIN Verlag, https://www.grin.com/document/936722
