Advanced data management project underway at Northeastern
When it comes to the world of scientific research, Sherlock Holmes said it best. It is: “Data! Data! Data!”
Indeed, Arthur Conan Doyle’s beloved character understood the value of tracking data – the cornerstone of science, human progress and, indeed, truth.
Today, scientific applications are producing an unprecedented amount of data, so much so that they are driving demand for better data management techniques. As a result, IT practitioners and researchers are working on more sophisticated methods to store, process, secure, and ultimately operate data in business and research environments.
That’s what Devesh Tiwari, an associate professor of electrical and computer engineering, hopes to do with new funding from the US Department of Energy. Tiwari is part of a multi-university and national laboratory team that recently received $2.7 million to design a software framework that would allow scientists in the field using data storage systems to more easily glean information from of “massive scientific datasets,” he says.
“We will design new storage interfaces and tools for autonomous and efficient data movement and metadata management,” says Tiwari.
Ultimately, this means designing techniques to facilitate and better manage the process of moving or extracting data from scientific instruments to storage systems, including cloud-based storage systems. As scientific instruments become increasingly complex and sophisticated, the need for efficient data storage systems becomes more important.
One example Tiwari gives to illustrate the project’s hopes are so-called “lattice light sheet” microscopes, which can produce exquisite images of biological specimens at “subcellular resolutions”. These cutting-edge devices produce far more data than they can store, which poses a significant problem for the scientists who use them.
As with any task on a computer, when data “takes flight” (moves electronically from source to storage destination), processing power is required, which translates into energy consumption and, therefore energy costs. Simply moving or transferring data can actually use a significant amount of power, up to 62% of overall computing power consumption, according to some estimates.
A question that researchers like Tiwari ask themselves is: can you also analyze data while traveling? That is, can computation be combined with data processing in a way that is energy efficient and in a way that helps computer scientists as they seek to gain insights from their own experiences?
“It’s quite complex to do,” says Tiwari. “But there are smart storage devices emerging where you can have computing power in the storage device itself.”
It’s something Tiwari has been thinking about for a while now: how to integrate high-performance computing with traditional storage devices.
“Nearly a decade ago, my co-workers and I were among the first to experiment with an interesting idea of embedding compute into flash storage devices to [high performance computing] data analysis tasks,” he says. “Such computer storage devices are now being prototyped and produced by companies such as Samsung.”
The question then becomes, he says, “can we enable IT people to take advantage of these unique storage devices with compute capability – and if so, how?” Tiwari and his colleagues demonstrated that they could expand such computer storage capabilities in the Department of Energy-funded project.
This money is just a slice of a larger disbursement — nearly $12 million in total — from the federal department to improve data management.
“The new data management and visualization capabilities developed by these projects will help make the most of the deluge of data generated by modern scientific experiments and simulations,” said Barbara Helland, Associate Director of Science for Advanced Scientific Computing Research. at the Department of Energy. , in a report.
Upgrading and refining data management methods would have a direct impact on a whole range of scientific fields, ranging from “materials science and chemistry…climate modeling and the development of new sources of clean energy, to new approaches to increasing energy efficiency and reducing energy consumption,” Department of Energy officials explained. said.
Collaborating universities include Lawrence Berkeley National Laboratory and the University of Illinois Urbana-Champaign. Tiwari thanks his students for their work in advancing these grant-worthy ideas.
“We are grateful for the support of [the Department of Energy],” he says. “I feel very fortunate to have wonderful collaborators at Lawrence Berkeley National Laboratory and the University of Illinois at Urbana-Champaign, and remarkably talented and creative students in my lab at Northeastern to make advance these ideas.”
For media inquiriesplease contact [email protected]