Advances in computation, including those in hardware, software, and algorithms, have enabled scientific research to progress at unprecedented rates. Machine learning permeates many aspects of society, including economic and social interactions^[8-11]. In 2020 alone, almost 9 billion compute hours have been used for scientific computing^[12], a pace of 24 million hours per day. Yet, the costs associated with large-scale computation are not being fully captured.

Machine learning (ML) models have grown exponentially in size over the past few years^[13], with some algorithms training for thousands of core-hours, and the energy consumption and costs required for ML training have become growing concerns^[14]. Traditional AI models, such as deep learning algorithms, rely heavily on complex neural networks that require vast amounts of data for training and learning. These models demand substantial computational power, which in turn consumes large amounts of energy^[15]. In natural language processing (NLP), Strubell et al. ^[16]found that designing and training translation engines can emit between 0.6 and 280 tonnes of CO2. While not all NLP algorithms require frequent retraining, algorithms in some fields are retrained daily or weekly, multiplying their energy consumption. As these models grow in both size and complexity, the demand for servers to process the models will grow exponentially (Figure 1)^[16].

Figure 1. Computing power used in training AI systems¹⁷.

The compute power used for training AI systems has exponentially increased in the era of deep learning^[17]. The exact total energy required for a single AI model is difficult to estimate as it must include the energy used to manufacture the computing equipment, create the model, and use the model in production. Researchers estimated that developing a generative AI model named BERT, which consisted of 110 million parameters, “consumed energy equivalent to that of a round-trip transcontinental flight for a single individual”^[18]. The training of GPT-3, which has 175 billion parameters, required 1,287 megawatts hours of electricity, which equates to 552 tons of carbon dioxide, equivalent of 123 gasoline-powered passenger vehicles driven for one year^[19]. That energy was just for getting the model ready to launch before any consumers started using it. The future development of larger models will substantially increase energy consumption and has raised concerns about the environmental impact of AI advancements.

To estimate the carbon footprint of nearly any computational task in a standardised and reliable way, Green Algorithms (www.green-algorithms.org; Figure 2), a generalisable framework and a freely available online tool, has been developed^[20]. This tool allows users to evaluate their computations or estimate the carbon savings or costs of redeploying them on other architectures.

Figure 2. The Green Algorithms calculator (www.green-algorithms.org).
 

The growing demand for AI-based applications, and the exponential growth in the use of computational resources due to AI advancements, has resulted in data centres accounting for an increasingly significant portion of global electricity consumption. As these data centres consume power, they generate heat which leads to additional energy consumption for cooling systems to prevent overheating. This vicious cycle contributes to an escalating energy demand that poses considerable challenges to both the environment and the power grid infrastructure. Widely cited forecasts suggest that the total electricity demand by the ICT industry will accelerate throughout the 2020s and that the percentage consumed by data centres will increase (Figure 3)^[21].

Figure 3. The forecast of electricity demand of ICT.
 

Figure 3 is an “expected case” projection by Anders Andrae, a specialist in sustainability ICT. Under his ‘best case’ scenario, ICT energy demand grows to just 8% of total electricity demand by 2039, rather than to 21% ^[21].