Why AI Inference Is Creating a Massive Data Storage Footprint

As AI inference scales, so does the amount of data organizations must store, manage and retrieve. Traditionally, discussions about AI infrastructure have focused on the compute-intensive process of training models. Training is limited: it starts, performs calculations and finishes. Inference, however, occurs continuously every time a model responds to a prompt.

This distinction is critical for AI infrastructure planning. Training is a project that can be budgeted and completed, while inference continues indefinitely. Every inference request can leave behind logs, cached context, rendering data, backups, safety records and other associated information.

AI-generated video can create 130 times more backend data than the output file

Consider AI-generated video. This workload can result in significant backend storage costs. Through internal measurements, WD found that an 8-second clip delivered to the user as a 3.97 MB 720p file leaves more than 500 MB of associated data on the backend. This includes logs, cached context, instant rendering, backups and safety records.

The backend data footprint is therefore approximately 130 times larger than the output file viewed by the user. For companies producing longer, higher-resolution videos, the backend footprint can reach hundreds of gigabytes per clip.

This data footprint is expanding rapidly. McKinsey projects that AI inference demand will grow at a compound annual growth rate of 35% by 2030. By comparison, training is projected to grow at 22%, while inference is expected to account for more than 40% of total data center demand by the end of the decade.

A recent WD-sponsored IDC whitepaper, based on a global survey of 763 IT and business decision-makers, found that nearly 95% of respondents reported an increase in the amount of data generated by AI during the past 12 months. Approximately 61% reported an increase of 25% or more.

“You can think of increasing storage as an optimization to keep the GPU busy,” said a senior product manager at a hyperscale cloud service provider quoted in the report. “The biggest challenge is making sure the GPU doesn’t sit idle waiting for data to be displayed. Hours of GPU idle time can be very expensive and very fast.”

In monetary terms, if storage spending increases by 25% year over year while computing budgets rise on a more predictable, cyclical basis, storage could eventually surpass the most visible budget line.

Adapt storage to the AI data lifecycle

Computing demands rise and fall with training cycles, product launches and hardware updates. However, even when computing demand declines, the amount of retained data does not return to normal. Model logs, synthesis outputs, inference results and other data continue to accumulate, leading organizations to retain more information for longer periods.

This changes the role of stored data. Information that was once moved to archives and rarely accessed may become input for future AI workloads. As more data is retained, retrieved and reused, storage must support not only greater capacity but also more dynamic access.

The same WD-IDC survey found that 96% of respondents expect to need faster archive searches to support inference or retrieval-augmented generation (RAG).

AI storage requires both performance and capacity

AI workloads require a combination of performance and capacity, and different storage technologies address different needs across the data lifecycle. For the vast amounts of data that must be stored and reused, hard disk drives (HDDs) provide a cost-effective, high-capacity foundation with a significant cost-per-terabyte advantage.