T h e s i s 

1.1        Overview

Practitioners have long recognised the economic and organisational impacts of poor quality information (Redman 1995). However, the costs of addressing the underlying causes can be significant. For organisations struggling with Information Quality (IQ), articulating the expected costs and benefits of improvements to IQ can be a necessary first step to reaching wider organisational goals.

Information Systems (IS) scholars have been tackling this problem since the 1980s (Ballou and Pazer 1985; Ballou and Tayi 1989). Indeed, information economists and management scientists have been studying this problem since even earlier (Marschak 1971; Stigler 1961). Despite the proliferation of IQ frameworks and models during the 1990s from IS researchers (Strong et al. 1997; Wang 1995) and authors (English 1999), the IQ investment problem has seen relatively scant attention within the discipline.

This research project seeks to develop and evaluate a comprehensive framework to help analysts quantify the costs and benefits of improvements to IQ. The framework should cover the necessary definitions, calculations and steps required to produce a business case upon which decision-makers can base a significant investment decision.

The level of abstraction should be high enough that the framework is generic and can apply to a wide range of situations and organisations. It should also be low enough that it can produce useful results to help guide decision-makers in their particular circumstances.

1.2       Background and Motivation

The research project partnered with Australia’s leading telecommunications company, Telstra Corp. The industry sponsor was responsible for the quality of information in large-scale customer information systems supporting activities as part of a wider Customer Relationship Management (CRM) strategy. As such, the quality of information about customers was the focus for this project. This grounded the research in a specific context (organisational data, processes, systems and objectives) but one that was shared across industries and organisational types. Most organisations, after all, have customers of one sort or another and they are very likely to capture information about them in a database.

A second agreed focus area was the use of automated decision-making at the customer level to support business functions such as marketing campaigns, fraud detection, credit scoring and customer service. These kinds of uses were “pain points” for the sponsor and so were identified as likely areas for improvements in the underlying customer data to be realised. Again, these functions are sufficiently generic across larger organisations that the framework would not become too specialised.

The third principle agreed with the industry partner was that telecommunications would not be the sole industry examined. While arrangements were in place for access to staff in the sponsoring organisation, it was felt important that approaches, experiences and practices from the wider community would benefit the project.

Lastly, the research project would not address the underlying causes of IQ deficiencies (eg. data entry errors, poor interface design or undocumented data standards) nor their specific remedies (eg. data cleansing, record linking or data model re-design). Instead, the focus would be on a framework for building the case for investing in improvements, independent of the systems or processes under examination. The industry partner was particularly interested in the benefit (or cost avoidance) side of the equation as the view was the costs associated with IQ projects were reasonably well understood and managed within traditional IS systems development frameworks.

Focusing the research on customer information used in customer processes struck the right balance between providing a meaningful context and ensuring the framework could produce useful results.

1.3        Outline of the Thesis

As the research project sought to produce and assess an artefact rather than answer a question, Design Science was selected as the most appropriate research approach. With Design Science, utility of a designed artefact is explicitly set as the goal rather than the truth of a theory (Hevner et al. 2004). So rather than following a process of formulating and answering a series of research questions, Design Science proceeds by building and evaluating an artefact. In this case, the framework is construed as an abstract artefact, incorporating models, measures and a method.

Before tackling the research project, some preliminary work must be completed. Firstly, further understanding of Design Science is required, especially how to distinguish between design as a human activity and Design Science as scholarly research. Further, a method for evaluating the artefact plus criteria for assessing the research itself must be identified. The philosophical position underpinning the research (including the ontological and epistemological stances) must be articulated, along with the implications for gathering and interpreting data. These issues are addressed in Chapter 2, Research Method and Design.

The third chapter (Literature Review) examines critically the current state of IQ research in regards to frameworks, measurement and valuation. The organisational context (CRM, in this case) and related measurement and valuation approaches (from information economics and others) are also examined.

In order to develop a useful artefact, it is necessary to understand what task the artefact is intended to perform and how the task is performed presently. This requires field work with practitioners who deal with questions of value and prioritisation around customer information. A series of semi-structured interviews was selected as the appropriate method here, yielding rich insights into the current “state of the art” including the limitations, difficulties and challenges arising from the existing practices (Chapter 4 – Context Interviews). Further, guidance about what form a solution to this problem could take was sought and this was used as the basis for practical requirements for the framework.

The theoretical knowledge from the Literature Review and the lessons from the Context Interviews were synthesised in Chapter 5 – Conceptual Study. This chapter is where the requirements of the framework are carefully spelled out and the core models and measures are proposed, defined and developed. An outline of the method is also provided.

To move from the development phases to the evaluation phase, Chapter 6 employs simulations and more detailed mathematical modelling to test empirically the emerging framework. This is done using a realistic evaluation approach, exploring the effect of synthetic IQ deficiencies on real-world data sets and decision-processes. This results in a number of refinements to the framework, the development of a supporting tool and illustration of the method.

Finally, Chapter 7 – Research Evaluation encapsulates the framework (Avison and Fitzgerald 2002) and evaluates it against a set of criteria (Hevner et al. 2004). This is where the argument is made that the framework qualifies as Design Science research.

1.4       Contributions of the Research

The research is an example of an applied, inter-disciplinary research employing qualitative and quantitative data collection and analysis. It is applied, in the sense that it identifies and addresses a real-world problem of interest to practitioners. It is inter-disciplinary as it draws upon “kernel theories” from reference disciplines in economics, machine learning and applied mathematics and incorporates them into knowledge from the Information Systems discipline. The collection and analysis of both qualitative data (from practitioner interviews) and quantitative data (from simulations) is integrated under a single post-positivist philosophy, Critical Realism.

The key contribution is the development, specification and evaluation of an abstract artefact (a framework comprising of models, measures and a method). This framework is grounded in an existing IQ framework, the Semiotic Framework for Information Quality (Price and Shanks 2005a) and extends the Ontological Model for Information Quality (Wand and Wang 1996) from the semantic level to the pragmatic. This model is operationalised and rigorously quantified from first principles using Information Theory (Shannon and Weaver 1949). The resulting novel IQ measures are used to identify and prioritise high-value candidate IQ interventions rapidly and efficiently.

At the core, this contribution stems from re-conceptualising the Information System as a communications channel between the external world of the customer and the organisation’s internal representation of the customer. The statistical relationships between external-world customer attributes and those of the internal representation can be modelled using the entropy measures developed by Shannon in his Information Theory. In this way, the research builds on an existing rigorous IS theory and integrates an important “reference discipline” (Information Theory) in a novel way.

The next step is the use of these internal representations of customer attributes to drive organisational decision-making. By employing Utility Theory to quantify the costs and benefits of customer-level decision-making, the costs to the organisation of mistakes can be quantified. By identifying how representational errors cause mistaken actions, the value of improving IQ deficiencies can be calculated. Here, Utility Theory is used as a “reference theory” to develop a novel normative theory for how rational organisations should invest in the IQ aspect of their Information Systems.

Finally, a systematic and efficient framework (comprising models, measures and a method) for identifying and measuring these opportunities is developed and assessed. This is important in practice, as well as theory, as it means that the time and resources likely required to undertake such an analysis are not unfeasibly demanding.

The contributions to Information Systems theory are:

·         the application of Utility Theory and Information Theory to address rigorously the value measurement problems in existing Information Quality frameworks,

·         the use of Critical Realism in Design Science research as a way to incorporate qualitative data collection (for requirements) and quantitative data collection (for evaluation) within a unified and coherent methodology,

The contributions to Information Systems practice are:

·         an understanding of how organisations fail to invest in Information Quality interventions,

·         a framework for producing financial models of the expected costs and benefits of Information Quality interventions to help analysts make the case for investment.

Further, the financial models produced by the framework could also be used by researchers as the basis for an instrument in Information Quality research. For instance, they could be used to compare the efficacy of certain interventions, to quantify the impact of various deficiencies or to identify Critical Success Factors for Information Quality projects.

 

 

Up: Contents

Next: Chapter 2 - Research Method & Design