Covering Disruptive Technology Powering Business in The Digital Age

Bringing Dark Data to Light
June 2, 2022 Blog


Authored By: Shamik Mehta, Director of Industry Solutions, Hitachi Vantara

From Energy Utilities to Amusement Parks, Industries Can Reap Huge Benefits from Computer Vision

At least every few years, utility companies release squadrons of drones into the skies to digitally photograph hard-to-reach equipment and check for wear and tear. Around the world, there are 3 million miles of high-voltage transmission lines – enough to go from the moon and back half-a-dozen times – and as much as 64 million miles of local transmission lines; that represents a lot of images, and even more data.

But what happens to this rich trove of visual information – or rather, what doesn’t happen – may surprise you. Typically, most images are archived without analysis. If a problem arises in the future, the images might be called up for historical comparison, or for evidence or regulatory needs, but the images are rarely used to detect and prevent the problem from occurring in the first place.

Such data is an example of what’s known in the IT realm as dark data – the unused and untapped information that companies have in vast amounts. By 2025, IDC expects that the amount of data worldwide will grow to 175 zettabytes – or 175 trillion gigabytes. Much of this data is unstructured form, captured by technology like sensors, the Internet of Things, and omnipresent cameras. More troubling is the fact that about 90% of unstructured data is never analyzed, according to IDC, and that 90% of all the unstructured data that exists today was created in the last two years

In the case of utilities, when images of transmission lines have been captured, they are analyzed manually, by experts. This is a laborious and time-consuming process that flies against the requirements of a quick-changing, digital world. It also requires investments by the utilities into hiring data scientists and other software talent.

To solve the dark data problem, for example, Hitachi Vantara’s Lumada Inspection Insights portfolio can derive insights from still cameras, radar, lidar, and images collected from near-orbit satellites. The technology can ingest and process thousands of image data per second, automatically detect, inspect, monitor, diagnose, and prognose asset health, and recommend ways to manage and lower risks.

While utilities and energy are an obvious sweet spot for such technology, many other industries from theme parks to manufacturing to buildings and infrastructure to light rail and transit to smart cities can benefit from it as well.

Consider a transmission substation, a high-voltage electric system facility that connects two or more transmission lines. These facilities can be small, with little more than a transformer and associated switches, or very large. They can be in the middle of a dense city or in a remote location. In any case, having people watch over these facilities on a constant basis is difficult.

Managing assets effectively has far-reaching implications. In the case of encroaching vegetation, sensors can immediately turn off nearby equipment which might be impacted. Utilities can also reduce the number of times maintenance teams have to go out – known as “truck rolls” – which in turn reduces carbon footprint, fuel, and O&M expenses, leading to a sustainable alternative.

Data and automation can also make a big impact on employee productivity. Take an automobile manufacturer that wants to automatically examine the final engine assembly of a vehicle. Currently, workers look for correct installation of the plastic outlays and covers, alignment issues, read oil gauges, and avoid human error. Artificial Intelligence can do this job faster and more efficiently, while humans oversee the task and have more time to devote to strategic duties.

A theme park company also wants to get more benefit from its dark data – in this case, quite literally dark. The company has flown drones through dark spaces to capture images of the rail systems that rides operate on. This information is being used to create machine-learning algorithms that will identify any defects, such as cracking weld joints or similar structural issues before they create worse problems.

Overall, the remedy to dark data is not a single data lake to rule them all or any single other act or method. The goal must be to have people, processes and technology that expand visibility, help to automatically understand, clean and integrate data, enable access to data both for analytics and operational workloads, and then catalog the data so it can be easily found. Your data stores can be one of your greatest assets – making use of them shouldn’t be a shot in the dark.