AI Data Center Network ABC: Building vs. Purchasing and Cost Chapter

The popularization of AI is driving the growth and investment momentum of AI data centers. Enterprises that were once almost completely monopolized by cloud providers are now expanding their AI data center coverage through private data centers, hoping to exert more control over AI workloads and applications in this way.

Fully utilize AI data center investment

According to IDC's forecast, by 2027, enterprise investment in AI data center switching equipment will grow to $1 billion, with a compound annual growth rate of 158%. During the same period, the exchange volume of cloud providers is expected to continue to grow strongly at a compound annual growth rate of 91.5%, with some enterprises planning to divest some training and inference workloads to private data centers and adopt a hybrid cloud strategy.

Whether to build your own private data center, purchase AI services from public cloud providers, or use a hybrid cloud model, ultimately, several key issues need to be considered:

Data sensitivity

Do you need to keep sensitive or proprietary data in your local private cloud, or do you need to comply with data sovereignty rules and limit the data to a certain geographic boundary? Technology, finance, government, and healthcare use cases are more likely to require private data centers to protect intellectual property or avoid litigation uncertainty.

Professional knowledge

How deep is your expertise in internal data science or networking? If there are suitable personnel, deploying a private data center is a good choice. If not, we can only develop or outsource professional knowledge.

Geographic location

Do you have sufficient facilities at the expected location to support the needs of the data center? The power consumption of each GPU in a large training cluster is 700W, which may require expensive power upgrades to existing facilities.
Alternatively, companies can choose to distribute AI clusters across multiple data center locations to ensure electricity is within budget. The inference performance, including RAG, may drive AI data centers further towards the edge, as small AI clusters are physically closer to users. For example, IoT applications deployed in production workshops.
Hybrid architecture supports enterprises to build within their capabilities and purchase outside their capabilities, thereby deploying AI functions for training, inference, and RAG in the most advantageous locations.

Time to market

How much time pressure is there for your product to be launched? If time is tight, public cloud services can accelerate product launch time and provide valuable time for proper planning of private data center deployment.
How is the progress of your AI transformation journey? Are you in the early stages and still need to conduct extensive experimentation to determine which method is suitable for your business? If that's the case, then choose public cloud. However, if you are committed to developing AI technology and have plans on how to use AI in different business segments, economic analysis typically guides you towards investing in private cloud infrastructure.

Enterprise Strategy

Like many cloud transformation projects, AI solutions typically start at the departmental level, creating AI cluster islands to address specific customer or operational challenges. As enterprises develop overall enterprise strategies with more unified and shared AI infrastructure, AI investment costs can be more effectively amortized, thereby aligning AI investments in private data centers with existing enterprise budgets.
Initially, for innovators and early adopters of AI technology, public cloud was the only option. Although public cloud remains an important component of most AI strategies, concerns about data security and cost are making private data centers and hybrid cloud racks mainstream.
At the recent virtual event on seizing AI opportunities held by Juniper Networks, customers, partners, and industry experts discussed their respective hybrid cloud use cases and strategies, including data security issues for financial institutions and strategies for balancing cost and performance using hybrid cloud.

Cost: Maximizing Return on Investment in the Expensive AI World

No matter which deployment mode is adopted, the high cost of AI deployment is no secret. The cost of AI is measured by budget, expertise, and time, all of which are subject to resource constraints. Professional knowledge and time are variable costs that vary from enterprise to enterprise, while the hard investment in AI is driven by the market and only faces the challenge of budget allocation.
The cost of each GPU server is about $400000, and the infrastructure cost of just one small AI data center can reach millions of dollars. However, perhaps we can breathe a sigh of relief now. AI frameworks such as PyTorch 2.0 eliminate tight integration and dependency on NVIDIA chipsets. This has opened the door for Intel, AMD, and other manufacturers to launch competitive GPU products, breaking the market pattern and bringing costs closer to normal.

When investing in private cloud or hybrid cloud models, application requirements, deployment patterns, and costs are important considerations. But companies don't have to fight alone. AI is a new thing for most enterprises, and AI infrastructure does bring complex challenges, but it is not magic - most of your current knowledge of data center networks is applicable.