This paper critically reviews approaches for the evaluation of investments in information
systems prior to their implementation. First, the ground for the review is prepared by
examining characteristics of evaluation, information systems and value. A classification of 54
evaluation approaches identified in English and German literature is then presented. Examples
of each class are reviewed and their advantages and drawbacks are discussed. Their use in
evaluation practice is analysed through the examination of empirical studies and directions for
future research are given. Today, a significant share of corporate funds is spent on the implementation, upgrading and
maintenance of an information system (IS). Recent studies show that in 2001 the IS budget of
companies worldwide accounted for an averaged 8.8% of total corporate revenues (cf. CSC,
2001). Consequently, a thorough evaluation of investments in information systems before,
during and after the implementation of a project is important. However, the normative
literature reports a great deal of difficulty in the appraisal of these investments (cf. Irani,
2002:11). Although IS evaluation has been an issue for both academics and managers for
more than three decades now, there are still serious concerns about how to select projects for
investments, how to control the development and how to measure benefits after the
implementation (cf. Farbey, 1999:189). This concern has been matched by increased research
activity which prevailed through two broad streams. The first stream aimed to directly
measure the payoff of IS investments for companies and came to mixed conclusions (cf.
Dehning and Richardson, 2002:8). The second stream addressed the question of how IS
investments can actually be assessed by decision-makers and particularly focussed on the
research of evaluation criteria, evaluation methods and the very nature of the evaluation
process (cf. Avgerou, 2000:570). Of late, several deficiencies in the field of evaluation methods have induced calls for in-depth
research. Academics have criticised the current state of the field as being immature and
fragmented (cf. Mahmood and Szewczak, 1999:491) and have thus demanded “an overview
of the whole panoply of evaluation methods, together with … the assumptions they depend on
...[in order to enable]... the identification of gaps.” (Farbey, Land and Targett, 1999: 205). [...]
Table of Contents
1. INTRODUCTION
2. THEMATIC BACKGROUND
2.1. Information Systems
2.2. Evaluation
2.3. Value
3. CRITICAL REVIEW OF EVALUATION APPROACHES
3.1. Antecedents to the Review
3.2. Effect-Assessing Approaches
3.3. Effect-Locating Approaches
3.4. Discussion of Usability of Approaches
4. EVALUATION IN PRACTICE
4.1. Review of Empirical Studies
4.2. Interpretation of Empirical Studies
4.3. Discussion and Future Research
5. CONCLUSIONS
A. Overview of Evaluation Approaches
B. Literature Search Strategy
Objectives and Core Topics
This thesis aims to critically review available methods for the ex-ante evaluation of information system (IS) investments. It seeks to answer how these investments can be assessed, their associated strengths and weaknesses, their usability in practice, and the overall role these evaluation approaches play in corporate decision-making processes.
- The theoretical foundation of IS, evaluation, and value as key criteria.
- A classification and critical review of 54 distinct evaluation approaches.
- The practical application of evaluation techniques based on empirical research.
- Identification of research gaps and limitations in current evaluation methodologies.
Excerpt from the Book
A great deal of the difficulties associated with IS evaluation stems from characteristics of information and information systems.
With regard to information, the determination of its value is particularly troublesome. One reason is that information has no intrinsic value because its value depends on the associated purpose (cf. Droste, 1986:90) and varies inter-subjectively. Defining the value of information in relation to the decisions that can be based upon it is also problematic due to intervening factors. For instance, the quality of managerial decision-making does not only depend on the quality of the underlying information but also on the individual competences of a manager to make use of this information (cf. Farbey, Land and Targett, 1993:13-14). Another reason is a fact known as “information paradox”: in order to assess ex-ante an information’s worth, its content has to be disclosed to determine the worth of the activities based on it after which there is no need to acquire the information anymore (cf. Walter, 1995:203).
Features of an IS account for further problems. First, the returns of investments in such systems can seldom be completely expressed in monetary terms because, being often of supportive nature, the system has a great distance to the cash flow generating activities of an organisation (cf. Huber, 1999:111). Second, the IS lifetime is hard to anticipate due to threats of technological obsolescence and changed organisational requirements (cf. Farbey, Land and Targett, 1993:13). Third, information systems are often difficult to distinguish from each other. Reasons for this are the growing integration of application systems (cf. Smithson and Hirschheim, 1998:164) and the evolution of systems through modifications and extensions over time (cf. Clegg et al., 1997:857). Fourth, the realised value after an IS project is seldom exclusively attributable to a particular IS investment, but is due to compound effects (cf. Anselstetter, 1984:10-11; Huber, 1999:112). For instance, major benefits and costs could not emerge from the IS investment per se, but more from the reorganisation induced by it (cf. Stefanou, 2001:206; von Dobschütz, 2000:446). Finally, IS investments often trigger not entirely predictable, organisation-specific processes of change (cf. Clegg et al., 1997:857). These processes complicate reliable estimations because they can prevent the realisation of expected benefits or can cause unforeseen costs (cf. Chircu and Kauffman, 2000:66-67). For instance, sales personal could hinder the value realisation of a CRM system by resisting to maintain the necessary data.
Summary of Chapters
1. INTRODUCTION: Outlines the significance of IS investment evaluation, highlights current research deficiencies, and defines the core research questions of the thesis.
2. THEMATIC BACKGROUND: Delimits and discusses the three central concepts for this work: information systems, the evaluation process, and the multifaceted notion of value.
3. CRITICAL REVIEW OF EVALUATION APPROACHES: Presents a classification of 54 evaluation methods, reviews representative approaches in detail, and examines their theoretical usability.
4. EVALUATION IN PRACTICE: Analyzes empirical studies to understand how IS evaluation is actually conducted in organizations and the role evaluation approaches play in practice.
5. CONCLUSIONS: Summarizes the thesis, highlighting the discrepancy between existing evaluation theory and practical application while providing recommendations for future research.
Keywords
Information Systems, IS Evaluation, Ex-ante Evaluation, Investment Appraisal, IS Value, Decision Support, Financial Approaches, Indicator Approaches, Multi-criteria Approaches, Stakeholder Management, Productivity Paradox, Organizational Change.
Frequently Asked Questions
What is the primary focus of this work?
This work provides a critical review of approaches for the ex-ante evaluation of investments in information systems, analyzing both their theoretical basis and their practical application.
What are the central thematic fields covered?
The thesis covers the definition of information systems, the nature of the evaluation process, the concept of value in an IS context, and a systematic classification of various evaluation methods.
What is the primary goal or research question?
The research addresses what approaches are available for ex-ante IS investment evaluation, their strengths and weaknesses, their usability in practice, and what role they play in actual investment appraisal.
Which scientific methods are employed?
The thesis utilizes a comprehensive literature review of both English and German academic sources, combined with an analysis of published empirical studies on IS evaluation practices.
What is covered in the main body?
The main body classifies 54 evaluation approaches into different categories (e.g., financial, indicator, multi-criteria) and discusses their potential, limitations, and the challenges of applying them in real-world business scenarios.
Which keywords characterize the work?
Key terms include Information Systems, IS Evaluation, Ex-ante Evaluation, Investment Appraisal, and the analysis of value within organizational processes.
How do the authors define the "information paradox" mentioned in the text?
The information paradox refers to the difficulty of assessing the value of information ex-ante, as its content must be disclosed to determine its worth, after which the need to acquire the information no longer exists.
Why does the thesis highlight the "formal-rational" versus "interpretive" research perspectives?
These perspectives represent two different approaches: one focused on objective economic performance and efficient choice, the other on the social and subjective nature of evaluation within a political organizational context.
- Quote paper
- Sascha Walter (Author), 2003, Approaches to the Ex-ante Evaluation of Information Systems: A Critical Review, Munich, GRIN Verlag, https://www.grin.com/document/15706
