Covering Disruptive Technology Powering Business in The Digital Age

Eric Brewer and colleagues from Google GOOGL -0.62% presented an interesting white paper at the Usenix FAST conference in Santa Clara, CA CA -1.35%. They argue that with the growth in portable devices and services in the Cloud, hard disks will be deployed primarily as part of large storage services housed in data centers. As a result of the growth of data center data on hard disk drives Google advocates some changes in their design.

Youtube users upload over 400 hours of video every minute. At one gigabyte per hour this results in a petabyte (PB) of new storage every day. As the figure below shows the trend for video uploading is increasing exponentially by about 10X every 5 years.

Nearline enterprise hard disk drives are the primary HDDs used in data centers and in servers. Google argues that there are changes needed in these drives to optimize them for large-scale data centers and services. There are three differentiators for HDDs used in data centers. The first is a focus on the aggregate properties of the collection of HDDs. The second is a focus on tail latency derived from the use of these hard disk drives providing live services. The third important factor is the various security requirements from storing someone else’s data.

Disk drives for data centers should focus aggregate satisfaction of five key metrics. These are higher input output (I/O) per second IOPS, limited primarily by the HDD seek time, higher total capacity, lower tail latency, security requirements and lower total cost of ownership (TCO).

Since the data in a collection of HDDs must be replicated on multiple drives to provide data protection there could be trade offs in for instance the Bit Error Rate (BER) of the HDDs if that could result in higher capacity or better tail latency. Big data centers generally avoid doing in lower levels what they will do anyway at higher levels so some functions performed within the HDD might be performed at the system level instead.

A data center will optimize its storage for the overall balance of IOPS and capacity, using a mix of HDDs, SSDs and RAM. Google says that the cost of storage for HDDs versus SSDs that have enough endurance for data center applications favors the HDDs in the cost per capacity. Google would like the HDD companies to improve the ratio of IOPS/GB, which is difficult for conventional HDDs. One of the outcomes of this change is to improve the read tail latency, that is the time that it takes to read most of the data for a live service, like YouTube.

Some particular things that Google suggests to create better HDDs for data centers include developing alternative form factors (3.5-inch HDDs are common today) that provide trade-offs in TCO, IOPS and capacity. This might include higher z-heights, allowing more disks in a drive, as well as sizes different from the 3.5-inch and 2.5-inch HDDs that are common today. They also suggest building HDDs that allow parallel accesses to data that requires multiple actuators for heads to write and read on disks.

This article was originally published forbes.com and can be viewed in full here

(0)(0)