Good Contents Are Everywhere, But Here, We Deliver The Best of The Best.Please Hold on!
Your address will show here +12 34 56 78
Need to move large datasets for Big Data, IoT, and ML/AI adversely impacts application performance and storage/networking/server costs, according to a recent survey from NGD Systems and G2M Research.

 

Irvine, Calif. – December 12, 2017:

 

G2M Research, an analyst firm covering the Non-Volatile Memory Express® (NVMe) marketplace, today released the results of its recent survey on the need for “Intelligence storage” for applications with large data sets. The survey, sponsored by NGD Systems, was conducted across 112 respondents from organizations involved in Big Data, artificial intelligence/machine learning, and Internet of Things (IoT) applications.

 

The purpose of the study was to gauge whether the movement of large data sets across existing processing and storage architectures negatively impacts the cost and usability of the data by applications. The results of the survey show that existing compute and storage architectures adversely impact the performance and cost of these applications, and that new architectures are needed if these applications are to continue to scale in size and capabilities.

 

“Datasets for applications such as Big Data, AI/ML, and IoT continue to grow at an exponential rate,” said Mike Heumann, Managing Partner, G2M Research. “Our research study shows that the majority of users in these application spaces are very concerned about how this growth will impact their ability to use these applications over the next 12 months. The majority of these end-users also believe that new approaches like processing data within storage devices will be necessary to overcome existing data movement bottlenecks.”

 

The movement of very large data stores is increasingly critical for real-time analytics in a variety of applications. However, this data movement is not without cost or impact. The key findings of the survey include the following:

 

1.    52% of respondents consider the movement of large data stores between storage systems, storage devices, and servers will be a significant problem for their organization either today or within the next 12 months.

 

2.    92% of respondents expect that data movement will adversely impact their organization, with 62% responding that it will impact server, networking, or storage costs, 48% saying it will impact application performance, and 29% saying it will limit the way data can be used.

 

3.    Over 79% of respondents believe that current processing/storage architectures will not be able to handle the amount of data in their industry in the next 5 years.

 

4.    64% of respondents believe that processing or preprocessing data inside storage systems/devices could help solve the data movement problem.

 

G2M Research has produced a report and infographic summarizing the data from the survey, which is available at http://g2minc.com/research.

 

“As the capacity of SSD drives and the number of SSDs within servers continue to increase, moving the data out of these drives into the CPUs will be become exponentially harder and more cumbersome” said Nader Salessi, President and CEO of NGD Systems. “The G2M Research survey clearly illustrates the issues that large data sets present to application architects for Big Data, IoT, and AI/ML, among others. In-situ processing like that of the NGD Systems Catalina 2 SSD provide a compelling alternative to moving large amounts of data between storage systems, storage devices, and servers/CPU complexes.”

 

One of the most promising concepts to address the storage-CPU bottleneck is the use of in-situ processing within storage devices. In-situ processing revolutionizes the deployment of a variety of applications that today require huge clusters of expensive multi-socket servers with large amounts of RAM. By significantly reducing the amount of data that has to be moved between storage systems/devices and servers/CPUs/GPUs, in-situ processing within NVMe flash solid-state drives (SSDs) can significantly reduce network size/complexity, CPU/GPU workload, and power consumption for applications utilizing high number of IOs.

 

NGD’s Catalina II NVMe SSD enables the capability of in-situ processing, and is the first product to help reset the CPU/GPU-storage gap and improve the data center TCOs. NGD’s NVMe SSDs also has the industry’s highest capacities and lowest power per TB (W/TB).

_____________

Source: http://www.ngdsystems.com/pr_20171212.html

0

 
Modern applications like big data analytics, facial recognition, IoT and video streaming, as well as next generation applications like artificial intelligence and machine learning place unique demands on both the compute and storage infrastructures. Most of the modern and next generations operate against a vast store of data. The application makes a query against that data set, which is processed in some way yielding the answer. They all require vast amounts of data to be stored and then count on compute to identify the subset of data the application needs to respond to the queries.

The Problems with Moving Storage Closer to Compute

The most common strategy to addressing the challenges presented by modern and next generation applications is to move storage closer to compute. The strategy is install storage inside each compute node and let the data reside there. Each query requires a large section of data, in some case all of it, is sent to the compute tier to identify the needed sub-section. Moving storage to the compute as a strategy does reduce network latency. However, while the CPU-to-media interconnect has improved with advancements like NVMe, there is still latency in the connection. There is also a complication of making sure the right process has access to the right data locally.

Moving Compute Closer to Storage

The first step in the process for most of these modern and next generation applications is to reduce the working set of data. Essentially, if the data set is the haystack, the application lives to find the needles in that haystack. If this is the case, it may make more sense to move the compute to the storage. That way the media can perform the data reduction or qualification before data is sent to the main compute tier. For example, a facial recognition program searching for Elon Musk dressed in black may send out a request to each drive for images of Elon Musk. Those images are sent to the main compute tier which does the more fine-grained search for Elon Musk wearing black. The first value of such an architecture is the compute for the environment scales, and does so at a very granular level, at the drive. The second value is the amount of bandwidth required to transfer the data to the main compute tier is greatly reduced since the drives are sending a much smaller subset of data instead of all of it. The third value is the compute tier does not have to scale as rapidly because the drives are doing more work.

Introducing NGD Systems

NGD Systems is announcing the availability of the industry’s first SSD drive with embedded processing. This is not a processor for running flash controller functions (it has that too) this is a processor specifically for off-loading functions from the primary applications. Developers of these modern and next generation applications will find adopting their applications to take advantage of the new drives relatively straightforward. The NVMe Catalina 2 is now available in PCIe AIC and U.2 form factors.

In-Situ Processing

While not a controller company NGD Systems does incorporate an “in storage” hardware acceleration that puts computational capability on the drive itself. Doing so eliminates the need to move data to main memory for processing, reducing bandwidth requirements and server RAM requirements. It also reduces the pace the compute tier needs to scale, which should lead to reduced power consumption.

Elastic FTL

Beyond onboard compute, the drives themselves also have top notch controller technology. The controllers (separate from the compute) on the NGD Systems SSD use proprietary Elastic FTL and Advanced LDPC Engines to provide industry leading density, scalability and storage intelligence. It enables support of the event changing availability of drive types including 3D TLC NAND, QLC NAND as well as future NAND specifications. The company also claims the lowest watt-per-TB in the industry.

StorageSwiss Take

Moving compute to the storage is the ultimate in “divide and conquer”, which may be the best strategy for applications needing to operate on large data sets. If every drive in the environment can reduce the amount of data that needs to be transferred into main memory for processing the environment becomes infinitely more scalable. Unlike many flash memory announcements, the NGD Systems solution should have immediate appeal to hyperscale data centers looking to improve efficiency while increasing response times. NGD Systems will show a demonstration of the technology at Flash Memory Summit 2017, August 8-10 in Santa Clara, CA. Vladimir Alves, CTO and co-founder of NGD Systems, will also make a presentation on August 10th, at Flash Memory Summit Session 301-C, entitled, “Get Compute Closer To Data”.
Source: https://storageswiss.com/2017/08/08/move-compute-closer-to-storage-ngd-systems/
0