A Guide to Enterprise Reporting

Publishing . . .

What Do I Need to Consider When Publishing Reports?

We continue to suggest reporting requirements, broken down by Technical Architecture, Publishing Environment and User Experience:

Publishing Environment

• Archiving - This is where you preserve a "snapshot" of the reports at a specific point in time. While everyone wants this, you need to ask them: Why? Is it for auditing purposes? Business continuity or disaster recovery? Performance reviews? Billing and legal disputes? Corporate history or biography? Understanding this will help determine what you archival needs are (eg source data, access logs or just reports?). This will also help work out the duration of different reports and datasets. You'll also need to check what you local laws are, and, if you're working for a tobacco company, your *ahem* "document retention policy". Burning CDs may not be acceptable if you need to keep data for seven years. And remember to regularly test your archives - the time you need to use them is the wrong time to find out the disk is bad.
• Data Quality - This is the hardest bit to get right, principally because no one wants to take responsibility for it. Worse, it means different things to different people! Best practice in this area involves getting people to accept roles like business owner and data custodian (or data steward). Most report users have a very unsophisticated view of the data supply process and are unable to distinguish between problems with the data and problems with the report. This can be very damaging to the credibility of the reports, especially when starting out. In turn, many people responsible for day-to-day operational (source) systems are uninterested in providing high-quality data for management reporting. Brokering agreements between these parties requires a blend of technical acumen and business savvy. Specifying the agreements in a contractual form (eg service level agreement or project dependency agreement) is next to impossible owing to the measurement problem. But, as a starting point, pick complete, accurate and timely, then go add your own - there are well over a hundred data quality dimensions here!
• Security - The point of security in this context is to ensure that the right reports are only seen by the right people. The three goals are confidentiality (no eavesdropping), authentication (no unauthorised access) and integrity (no manipulation). Typically, this is achieved with username/password login and access control lists (ie specifying whether each account allowed or denied access to a resource). You need to decide whether to do this on an application-level, report-level or dataset-level. Also, it is worthwhile doing a risk assessment to rate the different likelihood/severity of security breaches, for each report.
• Auditing - In general, the auditing requirements of management reporting are moderate. This is because they are not transactive: customers (and suppliers) usually aren't billed (or paid) based on management reporting numbers. However, depending on your organisation, management may attract bonuses and penalties based on the information and hence, the figures may be in dispute. As a starting point, you should store source data (eg log files) and business logic (eg perl scripts) separately, not just the results. This is because the way that certain statistics or metrics is derived may change over time. Also, continuous disclosure to managers will help get them comfortable with the figures, rather than getting an unexpected rude shock at the end of the reporting period. Another important link in the audit trail is user access logs (either web, email, PC etc) - who has seen what reports when? These may prove crucial during blamestorming.
• User Administration - Any reporting system worth its salt must allow for users to be added and removed and their permissions (to reports or data sets) updated. There are (broadly-speaking) two different ways of doing this: bureau and delegation. In the first model, a central authority takes applications, assigns accounts, handles user requests and queries and is generally responsible for the seamless matching users to reports. The other model has a root "super user" who then delegates certain permissions to other users - including permissions to create other user accounts. The limitation with the first method is that you really need to designate an admin person to run it all. The limitation with the second method is that it only works for very hierarchical organisations.
• Usage Monitoring - There's little point in rolling out a reporting system without any idea of whether or not its being used. Further, you may have security and audit requirements that mean report generation and accesses have to be logged. In any case, the best approach is to begin by defining success (or failure) criteria linked to the business case, and then looking for recordable events. Example events include report publication, report views (and other interactions), user account creation, logins and session times. Examples of relevant metrics might be total report count, total user accounts, views per report, views per account, time between logins and report views per session.
• Privacy - Clearly, the specific privacy requirements depend on your jurisdiction. But, broadly speaking, personally identifying information (such as names, phone numbers, credit card numbers, social security and other government IDs) needs to be handled with extreme care. This applies to both customers, staff, suppliers and other parties. Since most reports deal with aggregated data, it is probably only exceptional reports that include this. In fact, you really need to consider whether or not management needs to see this type of information as reports at all. For example, in some cases it is not lawful to use government identifiers as keys in your information systems. To manage these risks properly, you need to understand the implications of privacy breeches: loss of reputation, legal expenses, fines and penalties and possibly jail-time for senior managers! Regardless of where you are, the National Privacy Principles (in Australia) are an excellent framework to follow.
• System Support - Given the complexity of typical enterprise reporting environments, comprising of many interacting sub-systems, with data feeds and reports linking different parts of your organisation, someone needs to take responsibility for making sure it all happens each day. This might not be a full-time job, but report users and data suppliers need to have someone they can contact to handle enquiries, problems, change requests, troubleshooting and other issues as they arise. Getting management support to recognise the need for (and hence fund) such ongoing work is an important part of the business case for reporting projects. It's tempting to outsource this function to a commercial IT helpdesk, but given the organisational-specific nature of enterprise reporting, it's hard to see how this could be done with the required efficiency and effectiveness. A better option may be to have a central person/unit responsible who can call upon your in-house IT resources and subject-matter experts in the relevant units as required.