Potential applications of fast data technology

In the Industry 4.0 transformation, companies are increasingly investing in collecting and generating data from Internet of Things (IoT) networks. The development of 5G networking technology, artificial intelligence (AI) and machine learning contribute to accelerating the journey towards fully automated smart factories, smart cities, autonomous vehicles and many other areas.

Fast Data is the amount of data collected, transmitted and processed in real time from many sources, including: mobile devices, sensors, surveillance cameras. The number of sources can be up to hundreds, even millions of different devices. Fast Data is characterized by low data capacity but requires fast processing, low latency and continuity.

Applications and software play an important role in optimizing the ability to analyze and process Fast Data. In addition, the selection of hardware and storage devices must be suitable for the requirements of speed, latency, reliability, and can operate in harsh conditions and environments.

Fast Data currently has potential applications in two main areas, including autonomous vehicles and security surveillance drones.

Self-driving car

Autonomous vehicle systems require high performance and the ability to efficiently manage large amounts of high-speed data from sensors, infotainment systems, operating systems, and maps. Sensor data will make up the largest portion of on-board data storage, with most data coming from advanced driver assistance systems (ADAS) and vehicle-to-environment (V2X) communications.

Some of the storage will be for multimedia entertainment, games, voice AI applications and other features. In addition, the “black box” will become a legal and safety requirement by regulators.

Unlike current 2D navigation maps, high-definition (HD) maps differ greatly in their update rate, positioning method, and amount of data collected. Due to the high frequency of data updates, HD maps typically use real-time online updates via 5G mobile networks. Maps include static, semi-static, semi-dynamic, and dynamic layers; the base static layer is updated monthly or as needed.

Nand flash storage solutions play a key role in autonomous vehicle systems, providing fast boot-up and data storage capabilities to record critical events, store AI models and HD maps. Systems will need to process more data at high speed and with high reliability as vehicles become more intelligent.

Security Surveillance Drone

With an average flight time of around 30 minutes, the amount of new data that can be saved is at least 150 GB. With high-resolution maps, 3D models and AI integration, much more data will be generated.

MicroSD cards are still the primary storage device in drones. However, to provide higher storage capacity and speed, some specialized drones are implementing embedded eMMC, UFS, and even SSDs for some special purposes.

Future AI-enabled drones with autonomous capabilities and longer flight ranges will require higher storage capacities, along with requirements for enhanced HD map navigation, higher resolution captured data (4K video, images) and other data.

Fast Data storage solution of the future

In the above two applications, as well as in other areas of real-time data analysis, the gap between storage devices and data computation and analysis is tending to be closer together, even to local storage right on that device.

In addition to traditional storage forms such as embedded nand flash memory, memory cards and SSDs, some units apply storage solutions with new technology, larger scale, faster capacity and speed. A typical example is the WD Ultrastar Data24 storage using NVMe-over-Fabric (NVMe-oF) technology. This is a high-speed 100 Gbps storage system, with a capacity of up to 368 TB, specifically connected directly to edge servers to store and analyze Fast Data in real time.

Data has a lot of potential in the future. Most units are looking for ways to collect and create more value from it, including the combination of people and machines. Storage infrastructure is a part of leveraging the value of data. Choosing the right storage devices and solutions will help optimize investment and operating costs, opening up new business opportunities. In particular, NVMe-oF technology is considered the Fast Data storage solution of the future.

Doan Phong

Source