AI-Optimized Storage: The Technology Stack Feeding GPU Clusters

December 2025 Update: AI storage market growing from $36B (2025) to $322B by 2035. DDN EXAScaler delivering 4TB/s to NVIDIA Eos supercomputer. GPUDirect Storage enabling 40+ GB/s direct transfers; NVIDIA's November 2025 SCADA technology eliminates last CPU involvement. NVMe-oF growing at 27.8% CAGR as organizations extend PCIe-level latency across networks.

Storage bottlenecks idle GPUs. A single DDN EXAScaler implementation delivers four terabytes per second to NVIDIA's Eos supercomputer, feeding 18.4 exaflops of AI performance from 576 DGX H100 systems.¹ When GPUs cost tens of thousands of dollars per unit and training clusters reach thousands of accelerators, storage infrastructure that cannot maintain data throughput wastes millions in compute resources. The AI-powered storage market reflects the urgency, projected to grow from $36.28 billion in 2025 to $321.93 billion by 2035 at a 24.4% compound annual growth rate.²

Modern AI workloads demand storage performance characteristics fundamentally different from traditional enterprise applications. Training datasets measured in petabytes require sustained sequential throughput. Checkpointing operations must complete in seconds to minimize training interruption. Inference workloads generate unpredictable I/O patterns mixing small random reads with burst writes. Organizations deploying AI infrastructure at scale now evaluate storage systems based on GPU utilization metrics rather than traditional IOPS benchmarks.

NVMe-oF extends flash performance across the network

NVMe over Fabrics (NVMe-oF) enables high-performance workloads at scale by providing low-latency sharing of NVMe SSDs over high-performance Ethernet or InfiniBand fabrics.³ The protocol delivers performance similar to locally attached NVMe SSDs while allowing organizations to scale storage resources independently of compute, GPU, and memory allocation.⁴

Traditional storage protocols add milliseconds of latency through software stacks optimized for spinning disks. NVMe-oF eliminates these layers, achieving latencies in the tens of microseconds even when scaled to thousands of nodes using RDMA transports.⁵ TCP transports enable deployment over commodity Ethernet while still delivering substantial performance improvements compared to legacy NFS or iSCSI protocols.⁶

For AI infrastructure, NVMe-oF matters where every microsecond counts: training pipelines where GPUs idle waiting on data, checkpoint operations that must complete within strict time windows, and inference workloads requiring sub-millisecond response times.⁷ Published benchmarks show 351 GiB per second sequential reads with GPUDirect Storage integration, with latency reductions expected to lift effective GPU utilization by 2 to 3 times in I/O bound configurations.⁸

Industry adoption accelerates through 2025. Western Digital and Ingrasys established a partnership in May 2025 combining GPU server expertise with NVMe-oF and fabric-attached storage capabilities.⁹ Hitachi Vantara launched Virtual Storage Platform One Block High End in November 2025, a next-generation all-flash NVMe block storage solution designed for mission-critical and AI workloads.¹⁰ NVMe-oF systems forecast a 27.80% compound annual growth rate as organizations extend PCIe-level latency across networks to boost GPU utilization in distributed AI clusters.¹¹

GPUDirect Storage eliminates the CPU bottleneck

NVIDIA's GPUDirect Storage enables direct data transfer from storage to GPU memory without routing through CPU and system memory.¹² The technology removes a fundamental performance barrier in AI training pipelines where large datasets must flow continuously into GPU memory for processing.

Deep learning training involves frequent checkpointing operations where trained network weights save to disk at various training stages. By definition, checkpointing sits in the critical I/O path.¹³ A 100-billion parameter model generates approximately 800GB to 1.6TB per checkpoint, and training at scale with 16,000 accelerators requires 155 checkpoints daily.¹⁴ To maintain overhead below 5%, checkpoint operations must complete in under 28 seconds at that scale, shrinking to 4.4 seconds for 100,000-accelerator clusters.¹⁵

GPUDirect Storage addresses these requirements by enabling 40+ GBps direct transfer rates from storage to GPU memory.¹⁶ The Lenovo/NVIDIA reference architecture delivers 20 GBps per node with linear scaling capabilities, supporting LLM training, inference, and checkpointing functions.¹⁷ NVIDIA's November 2025 SCADA technology takes GPUDirect further by offloading even the storage control path to the GPU, eliminating the last CPU involvement in storage operations.¹⁸

Hardware implementations proliferate across the ecosystem. The HighPoint Rocker 7638D adapter enables GPUDirect Storage workflows with up to 64 GB/s bandwidth and predictable latency, particularly useful for large-scale training datasets.¹⁹ Storage vendors including DDN, Pure Storage, WEKA, and VAST Data certify their platforms for GPUDirect integration with NVIDIA DGX and HGX systems.

Parallel file systems power exascale AI

Parallel file systems distribute data and metadata across multiple servers, enabling aggregate throughput that scales with storage node count. Three platforms dominate AI and HPC deployments: Lustre, IBM Storage Scale (formerly GPFS), and WekaFS.

Lustre commands 41% market share in parallel file systems, followed by IBM Storage Scale at 17% and WEKA at 6%.²⁰ Each architecture optimizes for different workload characteristics.

Lustre excels in environments dominated by large sequential operations including scientific simulations and video rendering pipelines.²¹ The architecture prioritizes sustained bandwidth over small file handling, achieving near-linear performance scaling with additional Object Storage Servers (OSS) for bandwidth-intensive workloads.²² Lustre performs best with InfiniBand fabrics and powers most of the world's supercomputers. DDN's EXAScaler product packages Lustre with performance optimizations and enterprise management capabilities.

IBM Storage Scale provides superior performance in metadata-intensive operations.²³ The distributed metadata approach creates small files, modifies attributes, and structures complex directories more efficiently than Lustre's centralized metadata server architecture.²⁴ Storage Scale delivers consistent performance across varying I/O patterns and integrates into NVIDIA DGX SuperPOD reference architectures with GPUDirect support.²⁵

WekaFS targets AI/ML workloads specifically, designed from inception for NVMe SSDs rather than retrofitted from spinning disk architectures.²⁶ WEKA's distributed metadata eliminates the metadata server bottleneck that constrains legacy parallel file systems.²⁷ Benchmarks show WekaFS outperforming FSx for Lustre by 300% or more at similar capacities, with I/O latency sometimes less than 30% of competing solutions.²⁸ WekaFS supports pNFS, SMB, and S3 protocols, enabling multiprotocol access patterns common in AI pipelines.

DDN, Pure Storage, and VAST Data lead the vendor landscape

Three storage vendors dominate AI infrastructure deployments with products specifically architected for GPU cluster workloads.

DDN powers the highest-profile AI supercomputers. NVIDIA's Eos system incorporates 576 DGX H100 systems with 48 DDN A³I appliances delivering 12 petabytes of storage at four terabytes per second throughput in less than three racks using only 100 kW of power.²⁹ DDN announced Blackwell certification in March 2025, optimizing EXAScaler and Infinia 2.0 for DGX SuperPOD with DGX GB200 and DGX B200 systems.³⁰ A single DDN AI400X2-Turbo achieves 10x the minimum requirement of 1 GBps/GPU for both read and write operations paired with DGX B200, delivering up to 96% network utilization.³¹ DDN's partnership with Yotta for India's sovereign AI initiative deployed EXAScaler AI400X3 systems powering 8,000 NVIDIA B200 GPUs.³²

Pure Storage introduced FlashBlade//EXA in March 2025, projecting more than 10 terabytes per second read performance in a single namespace.³³ The platform targets customers running between one and tens of thousands of GPUs requiring 1 TB/sec to 50 TB/sec storage throughput.³⁴ FlashBlade//EXA's disaggregated architecture scales data and metadata independently using third-party data nodes, enabling massive parallel performance.³⁵ Pure Storage achieved FlashBlade//S500 certification with NVIDIA DGX SuperPOD, integrating the NVIDIA AI Data Platform reference design with GPUDirect Storage support.³⁶

VAST Data reached $2 billion in cumulative software bookings by May 2025.³⁷ The DASE (Distributed and Shared Everything) architecture delivers breakthrough parallelism for 100k+ GPU clusters at terabytes per second, eliminating AI data bottlenecks.³⁸ VAST claims over 50% lower total cost of ownership for demanding AI workloads through radical efficiency.³⁹ The platform supports exabytes of all-flash storage with industry-standard NFS, SMB, S3, and Kubernetes CSI access.⁴⁰ Microsoft Azure announced integration with VAST's AI Operating System in November 2025 for extending on-premises AI pipelines into GPU-accelerated cloud infrastructure.⁴¹

Checkpointing architectures balance speed and reliability

Model checkpointing creates the most demanding storage requirements in AI training. Checkpoint sizes scale with parameter count: approximately 8 to 12 bytes per parameter for mixed-precision training means a 100-billion parameter model generates 800GB to 1.2TB per checkpoint.⁴² Frequency requirements intensify with cluster scale, reaching checkpoints every 1.5 minutes for 100,000-accelerator deployments.⁴³

Modern training systems employ tiered checkpointing architectures. Fast-tier checkpoints write to node-local NVMe storage every few minutes. Mid-tier checkpoints propagate to shared file systems every 30 minutes. Durable checkpoints reach object storage like Amazon S3 only every few hours.⁴⁴ Asynchronous checkpointing allows training to continue while background processes drain local storage to global tiers.⁴⁵

Global checkpoint bandwidth requirements remain surprisingly modest even at scale. Analysis of 85,000 checkpoints across real-world systems found bandwidth typically well below 1 TB/s even for trillion-parameter models.⁴⁶ Checkpoint bandwidth per GPU decreases as model size grows because only a single data-parallel replica writes during checkpointing regardless of total cluster size.⁴⁷

Reported throughput varies significantly across implementations. Gemini reports 3.13 GB/s checkpoint throughput. Microsoft's Nebula (DeepSpeed) achieves 1-4 GB/s. These figures reflect the architectural tradeoffs between checkpoint frequency, storage tier, and acceptable training overhead.⁴⁸

Computational storage moves processing to data

Computational storage devices (CSDs) embed compute functions within storage hardware, processing data before transfer to reduce I/O bandwidth requirements.⁴⁹ The architecture proves particularly valuable for edge AI deployments facing limited computational resources, strict power budgets, and real-time latency requirements.⁵⁰

Advanced CSD applications include running databases, machine learning models, and analytics directly on storage devices. Some implementations support full Linux operating systems, enabling AI/ML inference on the drive itself.⁵¹ Edge deployments benefit from initial processing at the storage layer, filtering results before transmission to main processors.⁵²

The technology addresses edge AI's unique constraints. Running inference increasingly shifts onto edge devices to enhance accessibility, customizability, and efficiency.⁵³ Cisco launched Unified Edge in November 2025, an integrated computing platform bringing together compute, networking, storage, and security for real-time AI inferencing and agentic workloads at locations from retail stores to factory floors.⁵⁴

Emerging memory technologies support computational storage evolution. NVMe, 3D XPoint, and Resistive RAM (ReRAM) provide high-speed, low-latency alternatives to traditional storage, enabling more sophisticated near-data processing.⁵⁵ RRAM and MRAM technologies could reduce energy costs for frequent inference workloads at the edge.⁵⁶

Building the AI storage stack

Selecting AI-optimized storage requires matching technology choices to workload characteristics, scale requirements, and operational capabilities.

For training workloads with large sequential I/O patterns, Lustre-based solutions from DDN provide proven scalability and ecosystem integration. EXAScaler's certifications with NVIDIA Blackwell systems ensure compatibility with next-generation GPU infrastructure.

For mixed workloads combining training, inference, and analytics, WekaFS or VAST Data platforms deliver consistent performance across varying I/O patterns. The distributed metadata architectures handle small-file operations that strain traditional parallel file systems.

For organizations standardizing on NVIDIA infrastructure, GPUDirect Storage certification becomes essential. Pure Storage FlashBlade, DDN EXAScaler, and VAST Data platforms all integrate with NVIDIA DGX SuperPOD reference architectures.

For hybrid cloud deployments, NVMe-oF enables consistent performance models across on-premises and cloud storage tiers. Object storage integration through S3-compatible interfaces supports checkpoint archival and dataset staging workflows.

Introl's global engineering teams deploy AI infrastructure across 257 locations, configuring storage architectures from single GPU servers to 100,000-accelerator clusters. Storage selection directly impacts GPU utilization rates and total training time, making architecture decisions critical to AI project economics.

The storage performance imperative

Storage performance determines AI infrastructure efficiency more directly than many organizations recognize. GPUs costing $30,000 or more sit idle when storage cannot supply data fast enough. Training runs extending weeks or months multiply the cost impact of suboptimal storage throughput.

The market responds accordingly. All-flash arrays held 40.9% share of AI storage in 2024.⁵⁷ Pure Storage's FlashBlade//EXA targeting 10+ TB/s throughput, DDN's 4 TB/s Eos implementation, and VAST Data's 100k+ GPU cluster support reflect vendor recognition that AI workloads require fundamentally different storage architectures than traditional enterprise applications.

Quick decision framework

Parallel File System Selection:

Storage Vendor Comparison:

Key takeaways

For storage architects: - DDN delivers 4TB/s to NVIDIA Eos—benchmark for AI-scale storage - GPUDirect Storage enables 40+ GB/s direct-to-GPU transfer, bypassing CPU - Checkpoint sizing: 8-12 bytes/parameter, 100B model = 800GB-1.2TB per checkpoint - 16K accelerators require 155 checkpoints/day completing in <28 seconds each

For infrastructure planners: - Lustre: 41% market share, best for large sequential I/O - WekaFS: 300%+ faster than FSx for Lustre, NVMe-native architecture - NVMe-oF delivers tens of microseconds latency vs. milliseconds for NFS/iSCSI - Tiered checkpointing: local NVMe (minutes) → shared FS (30 min) → S3 (hours)

For strategic planning: - AI storage market: $36B → $322B by 2035 (24.4% CAGR) - Storage determines GPU utilization—$30K GPUs idle when storage bottlenecks - NVIDIA SCADA (Nov 2025) offloads storage control path to GPU entirely - Evaluate storage by GPU feeding capability, not traditional IOPS metrics

Organizations planning AI infrastructure investments should evaluate storage systems based on GPU feeding capability rather than traditional metrics. Sustained throughput during training, checkpoint completion windows, and multiprotocol access for diverse pipeline stages matter more than peak IOPS numbers. The storage layer that maximizes GPU utilization delivers the fastest time to model completion and the best return on compute investment.

References

1. DDN, "DDN Delivers Four Terabytes per Second with NVIDIA Eos AI Supercomputer," press release, 2024.

2. MarketsandMarkets, "AI-Powered Storage Market Size, Share, Trends and Industry Analysis," 2025.

3. Western Digital, "NVMe-over-Fabrics: Accelerating Data Center Innovation in the AI Era," technical brief, 2025.

4. RT Insights, "3 Reasons Why NVMe-oF is Redefining AI-scale Data Centers in 2025," 2025.

5. DataCore Software, "Breaking Storage Bottlenecks with NVMe-oF," blog, 2025.

6. Computer Weekly, "NVMe-over Fabrics: How NVMe-oF revolutionises shared storage," 2025.

7. DataCore Software, "Breaking Storage Bottlenecks with NVMe-oF," blog, 2025.

8. AceCloud, "Block Storage For AI Inference: 5 Benchmarks To Boost Performance," 2025.

9. Globe Newswire, "$125+ Bn Next-Generation Data Storage Markets - Forecasts from 2025 to 2030," December 5, 2025.

10. Globe Newswire, "$125+ Bn Next-Generation Data Storage Markets," December 5, 2025.

11. Mordor Intelligence, "AI-Powered Storage Market Size, Share & 2030 Growth Trends Report," 2025.

12. NVIDIA Developer Blog, "GPUDirect Storage: A Direct Path Between Storage and GPU Memory," 2025.

13. NVIDIA Developer Blog, "GPUDirect Storage," 2025.

14. VAST Data, "Optimizing Checkpoint Bandwidth for LLM Training," blog, 2025.

15. VAST Data, "Optimizing Checkpoint Bandwidth for LLM Training," blog, 2025.

16. MinIO Blog, "NVIDIA GPUDirect Storage and MinIO AIStor: Unlocking AI Performance," 2025.

17. Blocks and Files, "Storage vendors rally behind Nvidia at GTC 2025," March 18, 2025.

18. Blocks and Files, "Nvidia SCADA offloads storage control path to the GPU," November 25, 2025.

19. Network World, "HighPoint offers direct GPU-storage connection to speed AI training, inference workloads," September 2025.

20. HPCwire, "Comparing the Relative Performance of Different Parallel File Systems," based on Hyperion survey data.

21. Bacula Systems, "Lustre vs GPFS: Key Differences in Most Popular HPC File Systems," 2025.

22. Bacula Systems, "Lustre vs GPFS," 2025.

23. Bacula Systems, "Lustre vs GPFS," 2025.

24. Bacula Systems, "Lustre vs GPFS," 2025.

25. DDN, "DDN Simplifies Enterprise Digital Transformation with New NVIDIA DGX BasePOD and DGX SuperPOD Reference Architectures," 2024.

26. WWT, "High Performance File Systems for AI/ML," 2025.

27. WWT, "High Performance File Systems for AI/ML," 2025.

28. Gigaom, "Cloud Parallel File Systems," 2025.

29. DDN, "DDN Delivers Four Terabytes per Second with NVIDIA Eos AI Supercomputer," press release.

30. Storage Newsletter, "NVIDIA GTC 2025: DDN Expands AI Data Infrastructure for Enterprises and Announces Support for NVIDIA Blackwell-Based Systems," March 20, 2025.

31. DDN, "DDN Announces Support for NVIDIA Blackwell-Based System," March 2025.

32. DDN, "DDN and Yotta Power Sovereign AI for India with EXAScaler Deployment," 2025.

33. Pure Storage, "Pure Storage Introduces FlashBlade//EXA, the World's Most Powerful Data Storage Platform for AI and High-Performance Computing," March 11, 2025.

34. The Next Platform, "Pure Storage FlashBlade//EXA Boosts AI Performance, Scalability," April 1, 2025.

35. Pure Storage, "FlashBlade//EXA press release," March 2025.

36. Pure Storage, "Pure Storage Integrates NVIDIA AI Data Platform into FlashBlade to Fuel Enterprise AI Innovation," 2025.

37. Wikipedia, "VAST Data," accessed December 2025.

38. VAST Data, "AI Storage," product page, 2025.

39. VAST Data, "What is AI Storage and Why Does It Matter in 2025?" blog, 2025.

40. VAST Data, "VAST Data Platform Services," product overview, 2025.

41. Blocks and Files, "Azure gets VAST's AI OS as VAST builds out global AI operations data fabric," November 18, 2025.

42. Cudo Compute, "How checkpointing impacts AI infrastructure storage requirements and cluster size," blog, 2025.

43. VAST Data, "Optimizing Checkpoint Bandwidth for LLM Training," blog, 2025.

44. AWS, "Architecting scalable checkpoint storage for large-scale ML training on AWS," 2025.

45. arXiv, "DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models," 2024.

46. VAST Data, "Optimizing Checkpoint Bandwidth for LLM Training," blog, 2025.

47. VAST Data, "Optimizing Checkpoint Bandwidth for LLM Training," blog, 2025.

48. VAST Data, "Optimizing Checkpoint Bandwidth for LLM Training," blog, 2025.

49. Computer Weekly, "Computational storage: What is it and what are its key use cases?" 2025.

50. HPCwire, "The Inference Bottleneck: Why Edge AI Is the Next Great Computing Challenge," April 15, 2025.

51. Computer Weekly, "Computational storage," 2025.

52. VMware Octo Blog, "Why Computational Storage Makes Sense for the Edge," 2025.

53. Gcore, "Edge cloud trends 2025: AI, big data, and security," 2025.

54. Cisco Newsroom, "Cisco Debuts New Unified Edge Platform for Distributed Agentic AI Workloads," November 2025.

55. MDPI, "Deploying AI on Edge: Advancement and Challenges in Edge Intelligence," 2025.

56. MDPI, "Deploying AI on Edge," 2025.

57. Mordor Intelligence, "AI-Powered Storage Market," 2025.

URL Slug Options: 1. ai-optimized-storage-nvme-gpudirect-parallel-file-systems-2025 (primary) 2. ai-storage-ddn-pure-vast-gpudirect-lustre-weka-2025 3. gpu-cluster-storage-nvme-of-checkpointing-training-2025 4. ai-infrastructure-storage-parallel-file-systems-2025