Fifteen years back, it was difficult to do analytics, because data was trapped, either on paper or inside solitary datastores. Even if you wanted to do analytics, there were no right tools. Now, there is computational power, connectivity, a flood of analytical tools and most importantly more data, Big and small.

Data Stories

The age of Data Deluge has brought upon us a new age of requirements – Data stories. The new ‘Insight’ economy calls for an improved data discovery and analytic precision. No longer do analytic requirements squeeze themselves into reporting modules of design documents, waiting to be discovered at the very end of the design process.

Beyond User Stories

User stories act on the underlying datasets. Data Stories are told by looking through and beyond the existing data sets, looking beyond existing questions into a business problem to pose new questions. It is a discovery of identifying the business value associated with each incoming data item and how it impacts business decisions and strategic thinking.

To implement data stories, analytic requirements should be able to define:

Sources of raw data: Is the data you have enough? Where can additional data that adds more value come from - IoT data incoming from various sensors, POS, devices, sentiment data from social networks, weather, census, traffic, medical imaging, data from communication satellites, space stations and so on.

Security, Compliance and Regulatory factors: Data security is an important aspect that should not be overlooked when dealing with sensitive information. Most data is governed by various data protection laws like HIPAA, FCRA, ECPA etc. And they all come with the added scare of audits.

Platforms, tools and interoperability: Data sources in different forms and networks, data management tools, analytical tools all need to work in harmony to make sense. A myriad of open source tools with quirky names – ranging from kids’ toy names to something from Dr. Suess’s books appear far too often. Versions become obsolete quickly, posing a threat of breaking existing systems.

Data Selection:  Do we analyze too much and synthesize too little? A roadmap to analytics driven enterprise should include ways to detect anomalies, fraud, waste and abuse. What information is used, which questions are answered - the available technology should not define the plans, but KPIs should.

User-centric analytics design is data-driven metrics, while stakeholder-centric analytics design boils down to “hunches” of bigwigs. Hunches are important, but we need to have evidence to back up those hunches.

The analytics roadmap requires a clear definition for key performance indicators across different business units and departments. Achieving the drill down effect means to be clear on the particular job function, business process and organizational thinking.

Data scrubbing: External, free range data is not clean! Clean data is the foundation upon which predictive analytics, reporting, and workflow are built. To prevent the garbage in-garbage out scenario, data scrubbing tools should be in place to examine data for flaws. Typically, a data scrubbing tool is capable of correcting a no. of specific mistakes, such as adding missing zip codes or finding duplicate records.

Life cycle of Data: Real time business intelligence requires near zero latency while event based BI can do with higher latency rates. Principles of data warehouse architecture for latency and refresh rates of data, can be borrowed to apply to data for analytics to keep it relevant and useful.

Access rights: Clear role definitions that prevent unauthorized access of data, even for sole viewing purposes need to be in place. Also, all users do not consume information in the same manner. Dashboard design teams should do as much research as possible to fully understand the roles and needs of the users to support different levels of analysis functionality at different levels.

Appearance or Visualizations: Data analytics user bases often consist of corporate executives and heads of business units - a demanding audience! The proliferation of advanced data visualization capabilities has added new complications for BI dashboard designers to avoid overloading screens with flashy graphics. A dashboard should not simply be a visual version of the canned reports or present irrelevant information in a chaotic manner.

A certain super hero, who once got bitten by a spider said that, “With great power comes great responsibility.” Workflow based systems have visible problems – people will know immediately if something is wrong. But results of analytic/BI systems will be taken at face value. Users may not have the time and resources to verify the results, which makes it imperative for analytic requirements to be very precise, specific and the same time holistic and futuristic. The dashboard cannot lose credibility – it’s a zero tolerance zone.

(Shweta Katre, August 24 2015)