What does all this data mean, and how does SMC's technology provide for such a generational leap in energy efficiency? Unfortunately, for MLPerf® Training V4.0, we were the only submitters of power data, so we can only compare against commonly available data points currently. Our calculator uses our real-world benchmark results, allowing comparisons with other methods of building and running H100 SXM-based AI factories. For each MLPerf® test, you can compare our results with legacy systems.
Explore models
Llama2-70B-LoRA (1 node)
Input comparison data
We are comparing SMC HGX H100 SXM-based results. For an apples-to-apples comparison, we'll assume the same time to complete the benchmark for the comparison result as SMCs. Here, input assumptions for the average power per node, as well as a host data center. Hover over the labels for guides on input data.
Average power per node (kW):
?
We have experienced 9.50 kW, on average, when running HGX H100 SXM boxes in single and multi-node testing in air cooled environments, which is the bulk of the world's installed fleet of HGX H100 infrastructure today. NVIDIA's official documentation for DGX H100 notes a peak consumption of 10.20 kW.
Configure this input based on your preferences. For air cooled, a guide woud be between 9 and 11.50 kW.
9.5
Host data center PUE:
?
As surveyed by the Uptime Institute, the 2023 global average PUE was 1.58. In facilities larger than 30MW, this was a better but still substantial 1.44.
Many nations are now requiring new build data centers to be built to a maximum of 1.30 or in some cases 1.20, beginning at the end of this decade.
1.58
Results
We've simplified some of the inputs, including the energy grid carbon coefficient (we've assumed like-for-like running in Singapore) to make things easy. If you'd like to know more about how this all comes together, we'd be happy to help. Here are some of the key outputs of this comparison
SMC
Comparison
Variance
Node-only Total Power
Node-only Total Power
3.27 kWh
4.61 kWh
-29.03%
AI Factory Total Power: Node + Data Center
AI Factory Total Power: Node + Data Center
3.6 kWh
7.28 kWh
-50.55%
12-month Extrapolated Total Power
12-month Extrapolated Total Power
64,956 kWh
131,488 kWh
-50.6%
12-month Extrapolated Total CO₂
12-month Extrapolated Total CO₂
27,067 kg
54,791 kg
-27,724 kg
?
US Energy Information Agency. 2022 US average household energy consumption of 10,791 kWh per annum.
Node-only Total Power
AI Factory Total Power: Node + Data Center
12-month Extrapolated Total Power
12-month Extrapolated Total CO₂
US Energy Information Agency. 2022 US average household energy consumption of 10,791 kWh per annum.
US Environmental Protection Agency. EPA estimate of 4.6 metric tons per annum for a typical passenger vehicle.
US Department of Agriculture, Arbor Day Foundation. Estimate of 48 lbs per mature tree per annum of CO2 absorption potential