A Framework for Valuing the Quality of Customer Information
Gregory Hill
Submitted in total fulfilment of the requirements of the degree of Doctor of Philosophy
October 2009
Department of Information Systems
Faculty of Science
The University of Melbourne
This thesis addresses a widespread, significant and persistent problem in Information Systems practice: under-investment in the quality of customer information. Many organisations require clear financial models in order to undertake investments in their information systems and related processes. However, there are no widely accepted approaches to rigorously articulating the costs and benefits of potential quality improvements to customer information. This can result in poor quality customer information which impacts on wider organisational goals.
To address this problem, I develop and evaluate a framework for producing financial models of the costs and benefits of customer information quality interventions. These models can be used to select and prioritise from multiple candidate interventions across various customer processes and information resources, and to build a business case for the organisation to make the investment.
The research process involved:
· The adoption of Design Science as a suitable research approach, underpinned by a Critical Realist philosophy.
· A review of scholarly research in the Information Systems sub-discipline of Information Quality focusing on measurement and valuation, along with topics from relevant reference disciplines in economics and applied mathematics.
· A series of semi-structured context interviews with practitioners (including analysts, managers and executives) in a number of industries, examining specifically information quality measurement, valuation and investment.
· A conceptual study using the knowledge from the reference disciplines to design a framework incorporating models, measures and methods to address these practitioner requirements.
· A simulation study to evaluate and refine the framework by applying synthetic information quality deficiencies to real-world customer data sets and decision process in a controlled fashion.
· An evaluation of the framework based on a number of published criteria recommended by scholars to establish that the framework is a purposeful, innovative and generic solution to the problem at hand.
This is to certify that:
- the thesis comprises only my original work towards the PhD,
- due acknowledgement has been made in the text to all other material used,
- the thesis is less than 100,000 words in length, exclusive of tables, maps, bibliographies and appendices.
Gregory Hill
I wish to acknowledge:
· The Australian Research Council for funding the research project,
· Bill Nankervis at Telstra Corp. for additional funding, guidance and industry access,
· my supervisor, Professor Graeme Shanks, for his powers of perseverance and persistence,
· my mother, Elaine Hill, for her long-standing encouragement and support,
· and finally, my partner, Marie Barnard, for her patience with me throughout this project.
1
1.4 Contributions of the Research
Chapter 2 - Research Method and Design
2.2 Introduction to Design Science
2.4 Goals of the Research Design
2.5 Employing Design Science in Research
2.5.3 Infrastructure and Applications
2.6.3 Justify/Evaluate Framework
2.7 Assessment of Research Design
3.5 Customer Relationship Management
3.6 Decision Process Modelling
Chapter 4 - Context Interviews
4.4.3 Summary of Data Collection
4.5.1 Approach and Philosophical Basis
4.5.5 Summary of Data Analysis
4.6.5 The Context-Mechanism-Outcome Configuration
5.5.1 Organisational Processes
5.5.2 Decision-Making Functions
5.5.3 Information System Representation
5.5.4 Information Quality Interventions
6.5.1 Effects of Noise on Errors
6.5.3 Effects on Interventions
Chapter 7 - Research Evaluation
7.2 Evaluation in Design Science
7.3 Presentation of Framework as Artefact
7.4.6 Design as a Search Process
7.4.7 Communication as Research
8.3 Limitations and Further Research
Figure 1 Design Science Research Process Adapted from takeda (1990)
Figure 2 Design Science Research Model (Adapted from Hevner et al. 2004, p9).
Figure 3 IS Success Model of Delone and Mclean (DeLone and McLean 1992)
Figure 4 - PSP/IQ Matrix (Kahn et al. 2002)
Figure 5 Normative CMO Configuration
Figure 6 Descriptive CMO Configuration
Figure 7 Use of the Designed Artefact in Practice
Figure 8 Ontological Model (a) perfect (b) flawed.
Figure 9 Simplified Source/Channel Model proposed by Shannon
Figure 10 Channel as a Transition Matrix
Figure 11 Augmented Ontological Model
Figure 12 (a) Perfect and (b) Imperfect Realisation
Figure 13 Pay-off Matrix using the Cost-based approach. All units are dollars.
Figure 14 Costly Information Quality Defect
Figure 15 Breakdown of Sources of Costly Mistakes
Figure 16 Revised Augmented Ontological Model
Figure 18 Model if IQ Intervention
Figure 20 ID3 Decision Tree for ADULT Dataset
Figure 21 Error Rate (ε) vs Garbling Rate (g)
Figure 22 Effect of Garbling Rate on Fidelty
Figure 23 Percent Cumulative Actionability for ADULT dataset
Figure 24 Percent Cumulative Actionability for CRX dataset
Figure 25 Percent Cumulative Actionability for GERMAN dataset
Figure 26 Percent Cumulative Actionability for All datasets
Figure 27 High-Level Constructs in the Framework
Figure 28 The Augmented Ontological Model
Figure 29 Model of IQ Interventions
Figure 30 Process Outline for value-based prioritisation of iq interventions
Table 1 Possible Evaluation Methods in Design Science Research, adapted from (Hevner et al. 2004)
Table 2 ontological stratification in critical realism (Adapted from Bhaskar 1979)
Table 3 Guidelines for assessment of Design Science REsearch Adapted from(Hevner et al. 2004)
Table 4 Quality Category Information (Adapted from Price and Shanks 2005a)
Table 5 Adapted from Naumann and Rolker (2000)
Table 6 Subjects in Study by Strata
Table 8 Final Measure Sets (new measures in italics)
Table 9 Normative CMO elements
Table 10 Descriptive CMO elements
Table 11 Example of Attribute Influence On a Decision
Table 12 Outline of Method for Valuation
Table 16 - Decision Model Performance by Algorithm and Dataset
Table 17 gamma by Attribute and Decision Function
Table 18 Predicted and Observed Error Rates for Three Attributes, a0, c0 and g0
Table 19 Comparing Expected and Predicted Error Rates
Table 20 alpha by attribute and decision function
Table 21 Information Gains by Attribute and Decision Function
Table 22 Correlation between Information Gain and Actionability, by Dataset and Decision Function
Table 23 Information Gain Ratio by Attribute and Decision Function
Contents |
Next: Chapter 1 - Introduction |