Interoperability: Future enterprise data processing technology is essential now

0


Data is more important than ever, but the speed of data generation, along with the challenges of collecting, preparing, organizing, storing, managing, protecting and analyzing it, continues to challenge IT departments. In the opening keynote and in separate expert sessions at this month’s Interop Digital: Data Management, Storage & Disaster Recovery event, IT professionals and industry analysts discussed ways to meet these challenges.

“More and more, everything revolves around data in today’s world,” said Dennis Hahn, senior analyst at research group Omdia, in the opening speech of the Interop event. “IT has evolved from the mere operational brain of the organization to the data processing plant for business information. “

The number one challenge in processing business data today is the rate at which data keeps growing. Omdia’s research reveals that the data is growing at a compound annual growth rate (CAGR) of 22% over five years. The data is also changing. There is more unstructured data like emails, videos, photos, web pages, and audio files. There is also much more data produced at the edge, including data from sensors and other IoT devices, as well as data from mobile devices.

All of this disparate data has to go somewhere, and for most organizations that includes some type of cloud. With data and storage now spread across different locations, hybrid cloud seems to be the model of choice. The industry’s response, Hahn said, is to create a data fabric, or a holistic ability to manage the placement and movement of data across different locations.

This still leaves the question of where to store the data. Hahn recommended that organizations evaluate each application individually to decide whether it is best stored in a cloud or on-premises. In many cases, the best solution is one that allows data to move between on-premises and cloud environments. At the same time, many organizations are moving away from highly centralized and critical SAN storage for more distributed storage.

At the Interop Digital conference “Managing Data Flood with High Capacity Storage,” JB Baker, Head of ScaleFlux Product Management, noted that storage has come a long way.

“We’ve seen massive increases in the speed at which these devices can serve data, but a lot of them are really not optimized for the advancements we’ve had in storage technology,” he said. he declares. Many are still tied to the legacy of hard drives and have not taken full advantage of flash, especially NVMe flash, he added.

Distributed performance storage – storage that can deliver the performance needed to feed data to emerging functions such as real-time analytics, machine learning, artificial intelligence, and high performance computing (HPC) – will become also more important in the data processing strategies of companies.

“HPC compute implementations, for example, are designed to extract data, process it very quickly, and then send it out, with some sort of preview or whatever output,” Hahn explained. “To do that, you have to feed the data really fast. Traditional storage tends not to do a good job, especially at [acceptable] price levels. ”

New strategies for data collection and management

While advancements in storage technology are good catalysts for data storage, there are still many challenges regarding data collection and management. In the past, Hahn noted, organizations would simply pull data from their databases and put it in a data warehouse. But with more diverse and richer data streams coming from different places, things can quickly get confused. From the data warehouse or data lake, data sets need to be prepared and extracted for other uses, such as analyzes that provide insight.

So how can organizations sort through the vast amounts of data they dump into data lakes or warehouses to determine what is junk and what is valuable? Instead of transforming data before loading it into your storage, Hahn suggests transforming data after loading, which enables organizations to pick and choose the best data more efficiently.

Metadata – data about data – is also becoming increasingly important in this context, said Steve McDowell, senior data and storage analyst at Moor Insights & Strategy. McDowell suggests using a cataloging system that includes metadata. “Until you know what data you have, you don’t know what you can do with it,” he said.

Storage management absolutely needs to mature, Hahn said, and automation and artificial intelligence are helping to create smarter storage management. Infused with these advanced capabilities, software can now perform predictive diagnostics and performance optimizations.

Intelligent storage – the ability to intelligently analyze stored data – is important for both efficiency and security, said Dr Narasimha Reddy, storage site director for the Center on Intelligent Storage at Texas A&M University and participating in the “Elevating Data Management” panel. It is an effective way to eliminate abnormalities when they occur so that the reaction can be immediate. The same goes for fraud detection or supply chains. Businesses could also use natural language processing to analyze stored information related to voice calls, emails or video.

Say hello to computer storage and other advancements

While advances in traditional storage are important, there is still more going on behind the scenes of the enterprise data processing process. Computer storage, an architecture designed to offload some of the tasks of the processor, can alleviate bottlenecks and reduce data movement, thereby increasing efficiency. Computer storage drives, which Baker says will eventually replace regular SSDs, will go a long way in speeding up databases.

“For high performance databases, computer storage drives help by providing a better latency profile than regular NVMe SSDs, SAS or SATA SSDs. It’s much better than hard drives, ”he said. It can also save businesses money by storing more data per drive as it performs the compression task extremely efficiently.

In addition to compression, computer storage can perform application-specific tasks, such as filtering data. For example, you can filter to reduce the amount of data that you transfer over your network, which in turn reduces the amount of data that the processor must filter.

Another emerging technology is DNA storage, a type of dense storage ideal for reducing scalability and capacity issues. It is much denser than any other storage technology today and uses very little power, explained Western Digital vice president of strategic initiatives, Steffen Hellmold, during the “Managing the flow of data with storage” panel. high capacity ”. (Hellmold is a member of the DNA Data Storage Alliance.)

Scalability is key, Hellmold said, as many organizations will exceed the scaling capabilities of existing tools in the storage toolbox, especially HD and SSD. This is particularly relevant for data archives, which continue to grow exponentially over time.

This means that these tools need to be replaced with something that can increase orders of magnitude. This is where DNA storage comes in. Essentially, the process works by generating DNA strands from the four main compounds of DNA: adenine, cytosine, guanine and thymine.

There are many use cases for DNA storage other than archive management, including autonomous driving or video surveillance. If you need to prove that a car was involved in an accident or that a suspect committed a crime, you will likely be relying on huge amounts of data that has been collected over time, as well as all the data that have been collected over time. data used to train the algorithm.

The cheapest way to store large amounts of data for long periods of time can be DNA storage. But that’s just the tip of the iceberg. Hellmold gave another example that really brought out the possibilities. “You can encode and coat a medical tablet with DNA data storage that gives you the full history of what’s in the tablet and where it was made.”

With so many advancements and possibilities, it’s practically a full-time job to figure out how to move forward with processing, storing, and managing business data.

There is no easy answer, said Hahn; IT simply needs to find more efficient and effective ways to collect, prepare, organize, and store data. Data management, preparation, security and analysis come next. It’s a continuous cycle, but new technologies and methods continue to emerge, helping organizations make these difficult decisions.


Leave A Reply

Your email address will not be published.