The IT-market has been growing dramatically in the last 15 years. Software tries to solve complex problems of the real world. In addition, customers’ expectations concerning the quality of the software are rising steadily. Therefore, testing the software before it is delivered to a customer is a necessity. While testing software completely is impossible, the test department must use techniques in order to test critical parts in the software first. Furthermore, the test department has to provide best possible information about the quality of the software achieved through passing several tests. This requirement can be met only if testing is a structured and monitored process. This thesis presents the implementation of an approach which understandsstructured software testing as an integrated part of the software development life-cycle. The approach is based on several established models from software development, quality assurance, and software testing. These models are discussed together with different commercial, in-house, and Open Source tools used for the implementation of the approach. A framework is described for automated test execution, test results processing, archiving of test data, and web based test data management. An example is given where this framework has been introduced to a software company. Based on a specific quality approach the measurable quality improvement of the test process due to the introduction of the framework is presented.
Table of Contents
1 Introduction
2 General aspects of software development processes
2.1 Software development models
2.1.1 Waterfall-Model
2.1.2 V-Model
2.1.3 W-Model
2.1.4 eXtreme Programming (XP)
2.1.5 Conclusion
2.2 Quality factors and approaches
2.2.1 Capability Maturity Model Integration(CMMI)
2.2.2 Testing Maturity Model (TMM)
2.2.3 Test Process Improvement (TPI)
2.2.4 Comparison of TMM and TPI
2.2.5 Conclusion
3 Testing specific aspects of software development processes
3.1 The objectives and limits of testing
3.2 The Test plan
3.2.1 Strategical parts of the test plan
3.3 Test plan defined in IEEE 829
3.3.1 Test plan identifier
3.3.2 Introduction
3.3.3 Test items
3.3.4 Features to be tested
3.3.5 Features not to be tested
3.3.6 Approach
3.3.7 Item pass/fail criteria
3.3.8 Suspension criteria and resumption requirements
3.3.9 Test deliverables
3.3.10 Testing tasks
3.3.11 Environmental needs
3.3.12 Responsibilities
3.3.13 Staffing and training needs
3.3.14 Schedule
3.3.15 Risks and contingencies
3.3.16 Approvals
4 Concrete implementation of structured testing in a company
4.1 The starting point
4.2 Analysis of the current test activities
4.2.1 Test maturity matrix of testing at the starting point
4.3 The way to structured testing
4.3.1 The Test Browser
4.3.2 Storage management
4.3.3 Archiving process
4.3.4 Test automation
4.3.5 Test database
4.3.6 Build management
4.3.7 QaTraq - A web based software test case management tool
4.4 Conclusion
Objectives and Topics
This thesis aims to demonstrate the successful implementation of a structured test process within a company. The central research objective is to transition from unstructured, ad-hoc testing to a managed, efficient, and automated environment using established quality models and tools, thereby improving product quality and reducing development costs.
- Comparison of software development and quality maturity models (CMMI, TMM, TPI).
- Analysis of current test activities and identification of process weaknesses.
- Implementation of automated testing frameworks (Perl, Squish, Unit tests).
- Establishment of a centralized web-based management platform (Test Browser, QaTraq).
- Optimization of storage, archiving, and build management processes.
Excerpt from the Book
3.1 The objectives and limits of testing
A popular belief is that a tester can test a program completely. This section explains why this belief is not true and will not come true in the future. To illustrate the limits of testing a short example is provided which demonstrates the limits in a few words. Furthermore, this section explains why testing is important even if the test of the whole program is impossible.
Following Kaner et.al [KFN99], three reasons describe why testing a software completely is impossible:
• The domain of possible inputs is too large to test
• There are too many possible paths through the program to test
• The user interface issues (and thus the design issues) are too complex to completely test.
The following example comes up with a trivial program also provided in Kaner et.al [KFN99, chapter 2]. The program simply adds a pair of one- or two-digit numbers. Thinking about the possible inputs in this program results in the following facts:
• 39,601 different pairs of numbers are valid inputs
• If the program is enhanced to accept 4-digit numbers, the valid pairs of numbers will raise to 399,960,001.
However, testing the valid inputs possible only, will exceed the estimated costs for testing in a project. In addition, a tester must pass the following tests in order to reach the impossible target of testing a program completely:
• Test all invalid inputs
• Test all edited inputs
• Test all variations on input timing
• Test all possible paths the program can take
Summary of Chapters
1 Introduction: Summarizes the necessity of structured software testing to ensure quality and control costs in IT projects.
2 General aspects of software development processes: Explains software development models and quality approaches like CMMI, TMM, and TPI to measure and improve test process maturity.
3 Testing specific aspects of software development processes: Discusses the inherent limits of exhaustive testing and details the structure of a professional test plan according to IEEE 829.
4 Concrete implementation of structured testing in a company: Details the practical transition to a structured, automated test environment, including test automation, build management, and test case management via QaTraq.
Keywords
Software testing, structured testing, TPI, TMM, CMMI, test automation, build management, IEEE 829, test plan, quality assurance, software development lifecycle, QaTraq, Perl testing, Squish, unit testing
Frequently Asked Questions
What is the core focus of this thesis?
The thesis focuses on transforming unstructured, ad-hoc testing practices into a systematic, structured, and partially automated test process within a specific corporate environment.
Which quality models are central to this research?
The work primarily explores the Capability Maturity Model Integration (CMMI), the Testing Maturity Model (TMM), and the Test Process Improvement (TPI) model.
What is the primary goal of the implemented test process?
The goal is to improve the measurable quality of software products while optimizing the time and budget allocated to testing activities.
Which scientific methodology is applied?
The author uses a maturity assessment approach, identifying existing process gaps using TPI, followed by a targeted, step-by-step implementation of improvements and automation.
What topics are covered in the main section?
The main part covers the analysis of initial testing weaknesses, the implementation of automated testing frameworks, the setup of a web-based Test Browser, and the deployment of a test management tool (QaTraq).
What characterizes the keywords used in this work?
The keywords reflect the intersection of software quality assurance, process improvement frameworks, and practical automation tools for software testing.
How does the author verify the effectiveness of the changes?
Effectiveness is verified through the Test Maturity Matrix, which maps the improvements achieved over a one-year period in various key areas of the test process.
How does the thesis handle the limitation of software testing?
It explicitly acknowledges that complete testing of software is mathematically and practically impossible, thus emphasizing risk-based testing and the prioritization of critical software components.
How is the Test Browser utilized in the company?
It serves as a centralized, web-based platform that provides developers and testers with immediate access to test results, build reports, manuals, and metrics from any location.
- Quote paper
- DI (FH) Alfred Leithold (Author), 2007, Structured testing in practice, Munich, GRIN Verlag, https://www.grin.com/document/84574
