Like the sprouting high-rise buildings and skyscrapers in metropolitan areas around the world, the amount of data generated by the digital revolution is simply incredible.
Some people estimate that it doubles every year.
A distribution company, for example, has to collect sales data from all of its branches on a daily basis. A credit card issuing bank has to keep track of every purchase made by its cardholders. A telco company has to be able to provide its customers with details on the calls that they make.
And the data growth that has to be maintained will continue to accelerate. Government regulations, for example, require that a patient's data is maintained for many years even after his death. Online activities such as social networking also contribute to the explosion of data as users share videos and photos with groups of friends and relatives.
"Soon we will be talking about exabytes of data," said Hubert Yoshida, vice president and chief technical officer of Hitachi Data Systems (HDS), during the Hitachi Data Systems Inspirational Tour 2008 on July 17.
The second generation hard disk can still be seen showcased at Hitachi Data System research and development center in Odawara. It was built in 1985 and had a capacity of 1.5 GB. (JP/Zatni Arbi)
One exabyte is one billion gigabytes.
Specializing in storage solutions, HDS manufactures hardware products, develops the applications to manage storage and provides services to their clients.
I was invited by the company to visit their research and development center in Odawara, around 30 minutes by Shinkansen train from Shinagawa, Tokyo.
Yoshida underscored the fact that, while the amount of data has become so staggering, the infrastructure built by other storage vendors has remained the same as it was 20 years ago.
This infrastructure is no longer adequate to handle all the data that has to be gathered and kept today.
In trying to accommodate the increasing amount of data, what enterprises usually do is add more storage. The good news is that, as we have witnessed all these years, storage capacity has also grown.
However, enterprises have found out that acquiring new storage devices is not the right answer. They have realized that they are not using their storage resources efficiently. "Typically, enterprises use only 20 to 30 percent of the storage capacity that they have," said Yoshida.
To add more storage will only lead to increased costs and complexities in storage management. It also means more electricity is required; meanwhile the world is trying to reduce power output and cooling requirements as green has become the color of the day.
An answer was required to address these problems.
HDS' flagship technology is in storage virtualization. In fact, HDS is the market leader in this area. This technology, enabled by the company's Universal Storage Platform, enables a business enterprise to manage its data storage resources, regardless of who makes the devices.
Using HDS' controller, it consolidates the physical storage devices from various different vendors and makes all the external storage look like a single pool of storage.
"With virtualization, an enterprise can increase the utilization of its storage capacity up to 50 to 60 percent," Yoshida added.
Another key technology developed by HDS is Dynamic Provisioning, which allows an enterprise to store data without having to immediately add new storage devices. It is more like "storage on demand".
To increase data storage efficiency further, data that is considered old or "stale" may not have to be stored in a high performance device. Typically, data that is already 60 days old can be archived as it is no longer active. Such data can be stored in a low-cost, low-performance storage system, thus reducing costs further.
The software, which embodies the company's service-oriented storage vision, manages the movement of data across the infrastructure without causing any disruptions.
Another way to increase efficiency is to eliminate duplication of the same data, both structured and unstructured. Take, for example, the collections of product photos that a company usually distributes to their employees for review. They end up in everyone's hard disk, and then the hard disks get backed up.
HDS develops the technology to reduce the duplication through an approach that is called "deduplication".
HDS, a subsidiary of Hitachi Ltd. -- which ranked number 48 in the list of Fortune 500 Global companies last year -- may not be a familiar name even to IT people. The fact is, quite a few of its executives hold patent rights. It is much like Cisco Systems back in the early 1990s, when we began to use the Internet.
Top enterprise IT providers such as IBM and HP have used HDS' products and solutions in their high-end offerings to their own clients.
Today, HDS has 21 sales offices in the Asia-Pacific region, and a few years ago it opened its local office in Indonesia.
