Covering Disruptive Technology Powering Business in The Digital Age

Chief Analyst Officer, Michael O’Connell, took some time off during the Strata+Hadoop Conference in Singapore, to talk to Big Community in a quick interview to get his thoughts on the Big Data driven initiatives in the region.

He sees a lot of initiatives going on in the region in terms of smart cities and IoT while working with the Singapore government especially with Singapore being a global leader in the smart city segment.

“In the last years Data Analytics has had a growth spurt. In part there’s been a bit of a rise on the engineering side with low cost sensors everywhere. The customer side of the house too has had much progress with online activity and customer analytics, tracking of transactions and so on. We have seen a lot of growth globally”, he shared.

However, he wasn’t impressed about how people are talking about big data or IoT. He believes the actual insights and advantage comes from the analytics and data science. He added that a lot of people struggle to get value out of big data. Many people believe that by pumping data into it, it will magically produce results.

Regions across the world, the US included, have been through the growth pains of using big data the wrong way and slowly learning how to get real value from it. This region has the opportunity to leap frog that part of the learning curve by extracting the lessons experienced by their elder counter parts and get to the value straight away.

“Big Data doesn’t convert data into actionable information”, he says, quoting an article he wrote recently, “big data doesn’t create value. Data Science does and it doesn’t have to be complex, expensive or big”.

The fascination that is around Big Data needs to be cut through to get to the real value.

An example of how to take stored data and put it into action to make it valuable, was by making a customer feel important. This he says can be done by looking at the customers buying habits and creating a pattern. When the pattern is then used to enhance the customers experience, that will create a loyalty effect that would encourage the buyer to spend more. This would be how data analytics and data science create the real value.

Touching on perishable data, he spoke about online customers who are there only for the duration of the purchase or only while their interest is held on the page. Once they have left the page, no amount of spamming will encourage their buying behaviour. Therefore, this data needs to be acted upon quickly to be made of any value.

He also shared about real-time surveillance data from equipment that produce energy, such as wind turbines. The data gathered through the sensors he says, allow analytical science to interpret and provide them the information on the status of the equipment, in minute detail, what needs to be adjusted for optimization or what parts need to be repaired, therefore saving a lot of time and money analysing outcomes after the fact.

There are many applications for these types of sensors, especially in smart cities such as in transportation, energy production as well as high-tech manufacturing. Through the engineering of these smart sensors, they can provide the much needed and accurate information that can be acted upon in real-time.

“If you don’t take action, then there’s no value. Having the best analysis under the sun, sitting in a report on some guy’s desk, and nobody doing anything about it, has absolutely no value”, he says.

So what happens once the data has been acquired. That’s where the data scientist comes. Bringing all the information together, wrangling the appropriate data, to surface the combination of analytics and business knowledge and finally generate value from it.

“It’s a 3 legged stool really”, Michael quips, between collecting, wrangling and putting it all together for business purposes.

To a question on democratizing data, so that the insights of analysis are made available to the decision makers, Michael says that there are two sides to that coin.

“The ability for the engineer or the marketer to directly access the data is very powerful. Because now you have people who understand the business, now being able to make a data driven action from it. That has great potential and I’m all for it. One of the challenges though, is where the combination between IT and business can really find some synergy and to combine that into what we call Governed Data Discovery.”

He shared an example of a company that tried decentralization. The business people were collecting data and doing the analysis, getting different answers from various different analysis. Different tools were showing different results on the same business questions. They were in complete chaos.

Although it is very powerful to have the tools in the hands of the engineers but there should be requirements for it to be governed with best practices and centralisation to prevent such a situation arising in a company.

Michael believes that the two worlds of democratization of data and the knowledge derived from IT experts, have to come together, rather than separated.

“It’s finding the sweet spot in an organization. You don’t want to have chaos and you don’t want to slow it down either. Every company has a slightly different balance depending on the size and the dispersion and different pockets of usage and where the data are.

“People are talking about Data Gravity. Often times, you want to bring the analytics to the data. You don’t want to have cumbersome processes pulling data out form where it is and into a data lake. In fact, we have a collaboration with IBM and we are putting Spark on the mainframe”, he explained.

He says the driving force around this collaboration is data gravity. It makes little sense to bring a massive load of mainframe data and move it to the lake, spending a lot of resources and time in the process.

“Data Gravity is an important consideration. It doesn’t have to be the mainframe example. Where are the data? How do we get the analytics and the data and co-locate in an optimal way?”.

Michael saw the conference as a good platform where there is a lot of hype around Spark and Hadoop. The marriage of the two he found, is interesting. With one being about speed and the other focusing on storage and volume. Although he sees it as more of a temporary collaboration.

He believes that as Spark matures, the data can be fed into it in an analytics and real-time layer that will be beneficial to users in a variety of ways. It will also further enhance how they will use and derive insights from Big Data Analytics.

(0)(0)