This page contains Frequently Asked Questions relating to my PhD research project, entitled A Framework for Valuing the Quality of Customer Information. If you would like to learn more about my project, I have a collection of relevant documents under my Research Documents page.
Q1. What did you do?
Research for the degree Doctor of Philosophy (PhD) at the University of Melbourne, Faculty of Science, Department of Information Systems. I started at University of Melbourne but transferred to Monash in August 2003 when my supervisor Prof. Graeme Shanks was promoted. After several years at Monash, he's now back at Melbourne Uni, where I returned in 2007.
Q2. So who paid for it?
Primarily, the taxpayers of Australia, through a scholarship from the Australian Research Council. Additionally, this research has a industry sponsorship from Telstra Corp. (an Australian telco) under the SPIRT program. They provide some cash and in-kind support to the project. Yes, I used to work for Telstra, and no, I'm not obliged to work for them afterwards, and no, Telstra doesn't own or control the outputs of this research.
Q3. What did this get you?
I have made an original contribution to knowledge, with resulting warm and fuzzy feeling. I also get to use the title "Dr", and forevermore pay double for a plumber. In practice, this is not a particularly important qualification for my work in the way that it is for, say, scientists or historians.
Q4. What is Information Systems anyway?
IS is an emerging study area. It is different to Computer Science or Software Engineering, and most universities now have a separate department for each. IS deals with the use of information by people within organisations and society, rather than the mechanics of how computers work. It's multidisciplinary, and uses many different lenses to examine these systems, such as psychological, economic, sociological, financial, ethnographic and philosophical. This is in contrast to pure disciplines, such as physics, which use the same lens to examine many different phenomena.
Q5. And you were looking at ...?
Information Quality. This is a sub-discipline that studies the quality or fitness for purpose of information. We all know what we want from our information: complete, consistent, reliable, concise, valid, relevant, current, accurate, credible, timely, usable, actionable, coherent etc. There are over one hundred such aspects of Information Quality identified in the literature, and at least a dozen frameworks for grouping them together. There are also dozens of firms providing tools and consultancy in helping organisations understand and remedy problems, as well as journals, websites, conferences and so on.
Q6. What is your lens then?
I'd define it as economic, or more specifically, decision-theoretic. This is a mathematical approach, based on Utility Theory and Information Theory - it's sort of a gamblers view, as it deals with pay-offs and probabilities. In terms of the conduct of research, I'm using Critical Realism as the philosophical basis and Design Science as the approach.
Q7. What real-world problem were you tackling?
What I call the Information Quality Investment Problem. A recent report suggests that Information Quality problems cost the US economy US$600 billion each year. This is a staggering amount of waste, several times over what the US spends on defence. Clearly, there are significant societal benefits to gouging these costs out of our economy. But, you need to spend money now to save money later. This is an investment, which requires an economic explanation.
Information Quality improvement projects are widespread, consume huge amounts of resources and are prone to failure. I'm interested in the economic justification of these projects. Most organisations - in part - justify initiatives with a business case: an argument for undertaking a particular course of action based on the consideration of the costs and benefits of alternatives.
This means you have to measure the value of information quality improvement before it happens, to prioritise initiatives. Also, you have to measure the value afterwards, to reward (or punish!) suppliers. How do you measure the value of, say, improving the consistency of information in a database? What about currency, or accuracy? How do you convince people that your rationale is sensible? Will people accept this if they think their Christmas bonus is riding on it? Clearly, to answer these questions requires consideration of the use that the information is being put to, which means I need to focus on a particular domain.
Q8. What domain were you examining in particular?
Customer relationships in mass-market, multi-channel, multi-product environments. This is the sort of relationship you enjoy with your bank, Tax Office, phone company, insurer, electricity supplier etc. If you ring up and get a call centre, then you're probably in one of those relationships.
I'm interested in how Information Quality affects these relationships, because they involve a large group of people being categorised into a very small number of treatments. Think about it: your bank may have a couple of million customers, but only a dozen different ways of treating them, in terms of correspondence they send, products they offer or limits they set. Somewhere in the bowels of the organisation, a computer is classifying people into different buckets based on the information the organisation has about itself, its customers and the market.
Q9. But isn't that just an affront to human dignity, and a violation of privacy? Why would you want to help organisations be better at being evil?
The short answer is: we have enough problems controlling deliberate evil like spam without having to contend with accidental evil like poor information quality. If your friend was denied a mortgage because she ripped off the video store, then I've got no sympathy. If she was denied a mortgage because the bank peeked at her uni exam results, then that's illegal and should be dealt with through the courts. If she was denied a mortgage because someone mis-typed a number into a computer, then that's a crying shame for everyone and we should make sure it doesn't happen.
Q10. What's your thesis?
In a nutshell, I think that the success of this customer classification depends on the quality of the information about the customer. If you think of these classifications as a series of guesses, then investing in information quality initiatives makes your guesses right more often: in effect, you're buying luck. This raises the question: what price would you pay for this luck? Like any gambler could tell you, it depends on the stakes, how lucky you are anyway, how long the luck lasts, and what else you could have done with the money (like bet with it!).
Q11. What was the end product?
In terms of my degree, a thesis of about 85,000 words accepted by anonymous external examiners as a contribution worthy of a doctorate. In terms of my project, an artefact (a set of documents and spreadsheets) that embodies a framework comprising of a model, metrics and a method for analysing these situations. The idea is that this method can be used by an analyst within any organisation to come up with justifiable recommendations on how to tackle these information quality initiatives. For example, how to prioritise projects, how to choose between competing vendors, how to work out penalty/bonus structures for contracts or staff, how to measure improvements over time or how to compare performance between organisations.
Q12. What research were you undertaking to support this?
I did a literature review, covering most of what's been written on these topics before. I also did a series of interviews with analysts, managers and executives from relevant industries who grapple with these things day-to-day. I wanted to understand what kinds of information quality activities they undertake, how they justify them, and what kinds of measures of success they use beforehand, during and afterwards, to help manage these projects. I also completed a conceptual study, synthesising elements of the reference disciplines into a framework, and a series of computer simulations backing up the quantitative analysis of the models and metrics.
Q13. Sounds interesting if fanciful - where can I find out more?
A good place to start is my collection of project documents. The Research Proposal contains a few dozen academic references. Also, my homepage lists some practitioner-oriented websites. Or you can email me with any further questions or feedback.