Covering Disruptive Technology Powering Business in The Digital Age

Companies are generating more data, and they are holding on to it for longer. However, this data is often then left alone and unused. Research in the Forrester Wave on Hadoop distributions found that, on average, between 60% and 73% of all data within an enterprise goes unused for business intelligence (BI) and analytics.

Having all this data and not making use of it is problematic for both IT and the business. With apologies to Samuel Taylor Coleridge, there is “data, data everywhere, nor any stop to think.”

One of the biggest challenges is that Hadoop is built for analysts and data scientists, rather than business users. The data that goes into a Hadoop instance can be useful to those business professionals, but it’s not easy to get that data out in a suitable format that they can work with.

Alongside this, there are other big data implementations that could also be used by the business. Operational data stores such as Cassandra or the other NoSQL databases could provide value if tapped, while the community around Apache Spark is growing massively too. However, you still need to be a data scientist or hard core data-analyst to write SparkSQL. Each of these big data platforms could be useful if analytics can be brought to bear on the data they create.

The challenge here is about more than simply reporting on what data is there.   Companies must understand what role each of those big data tools provides and how it feeds into the whole mix. Working across divisional teams, different silos of data and various big data platforms can lead to data remaining a technical project, rather than a business one.

One area perhaps most often misunderstood is that data in a data lake can also be in silos. Consequently, it’s very hard for any business user to find the right data in the lake and be confident in that data. Building up the knowledge base of users across the business and helping them make use of data is therefore essential for the future.

Making big data ready for the business to use

With all this data being collected, it’s important to consider how to put it to better use. Rather than looking at this as a technical project, data should be looked at as a business resource for departments to use. To get this started, look at where data is being collected by the departments in their applications, as well as where any big data stores may have information that can be used by those teams.

This preparation work around what data exists can then be used to look at what objectives those teams have. The likes of marketing and sales will focus on customer acquisition, while operations and procurement will look for efficiencies and ways to save money. The issue for IT is in making data understandable and useful for them in meeting those goals over time. Translating those business goals into key performance indicators (KPIs) can help, as data can then be used to influence the KPIs. The idea is to provide data in a way that makes sense for that individual, group or department, while maintaining consistency across this set of diverse audiences.

Doing this involves looking at where teams collaborate – for example, where sales teams take leads from marketing, or where operations asks procurement to buy goods and services. Comparing data from those different applications can provide opportunities to influence decisions that are made, while also putting big data to better use.

Making data part of the furniture

Whether companies appoint someone specifically to manage use of data within the business, or keep it as part of the CIO’s remit, the deployment of analytics has to change as big data continues to grow in importance. Giving people access to this data is a process problem, as the results can force changes in behaviour that are perceived as unnecessary or contrary to their experience.

The role of those team KPIs can be brought into play, as well. Discussing the results and how the business unit or department uses data currently can help show a path to better performance. This can also demonstrate that IT is aiming to solve the same problem as the department manager, which can lead to more trust between the individuals involved.

The issue here is when KPIs are based on data from multiple departments. This can lead to those teams feeling that they lack control over the outcomes. The approach should be to work with those teams so that the overall business process is understood first, then the analytics reporting or dashboard created. Data sources can be networked together and then the results given over to those who require access. By working on collaboration first, on the analytics side, each business team can be involved.

This makes it more likely that the analytics results will get used regularly. It’s also possible to run analytics projects alongside a control team that does not get access to the results. When the analytics data gets used and supports better performance, more individuals within the team will want access.

For BI, big data can be a vital part of the range of data sources that each business department uses. However, the aim should be to make users smarter in their daily activities. Focusing on business aims can help ensure that the right data gets analysed and provided to those users at the right time.

This article was originally published on www.computerweekly.com and can be viewed in full

(0)(0)