Oct 15, 2024

HaraAI — Understanding and Tracking AI's Environmental Impact

Written by:

Anu Sridharan & David Menezes

HaraAI is an initiative that tackles the environmental costs associated with AI systems. While we often consider the carbon footprint of traditional industries, the energy demands of AI and large language models have received less attention – until recently.

As artificial intelligence continues to advance and integrate into various aspects of our lives, it brings with it a set of challenges that extend beyond its immediate applications. One such challenge, often overlooked in discussions about AI's potential, is its environmental impact.

The rapid growth of AI technologies has led to increased energy consumption in data centers and computing infrastructure. This raises important questions about the sustainability of AI as it becomes more prevalent in our society.

This article explores the environmental implications of AI, focusing on energy consumption and carbon emissions. 

As artificial intelligence systems grow in scale and complexity, their energy consumption has become a significant concern. Kate Crawford and Vladan Joler highlight this issue in their essay "Anatomy of an AI System," noting that while large-scale AI systems consume vast amounts of energy, the specifics of these costs often remain unclear to the public.

Recent studies have shed light on the scale of this energy usage:

  • Cloud computing, which powers many AI systems, has emerged as a major carbon emitter. According to MIT research, its emissions now surpass those of the entire airline industry.

  • The energy demands of AI are growing exponentially. OpenAI researchers have observed that since 2012, the energy required to train AI models has doubled approximately every 3.4 months.

  • The environmental impact of AI development is substantial. Training a single large natural language processing (NLP) model can emit about 300,000 kg of CO2—equivalent to the emissions from roughly 125 round-trip flights between New York and Beijing.

Quantifying AI Energy Consumption: The Case of ChatGPT

To better understand the scale of AI energy consumption, let's examine the energy use of ChatGPT, a popular large language model:

A single ChatGPT query consumes approximately 2.9 watt-hours (equivalent to 10,400 Joules) of energy. This is 10-15 times more than a typical Google search, which uses about 0.3 watt-hours.

To put this into perspective, the energy used for one ChatGPT query could power a standard lightbulb for about 20 minutes. On a larger scale, with an estimated 10 million queries per day, ChatGPT's daily energy consumption reaches a staggering 104 billion Joules.

This amount of energy is equivalent to the daily power needs of either 1,000 US households or approximately 10,000 Indian households.

Factors Contributing to High Energy Consumption

Several factors contribute to the high energy consumption of AI systems:

  • Computational intensity: Training and running large AI models require extensive computational resources.

  • Data center infrastructure: The energy needed for cooling and maintaining data centers adds to the overall consumption.

  • Redundancy: To ensure reliability, many systems use redundant setups, multiplying energy usage.

  • Continuous operation: Many AI systems run 24/7, leading to constant energy demand.

Understanding these factors is crucial for developing strategies to mitigate the environmental impact of AI technologies. 

Measuring AI's Carbon Footprint: The Token Approach

We can leverage existing metrics—specifically tokens—to track AI's carbon emissions. A token is a unit of text that measures computational complexity. The more complex the computation, the more tokens required. This approach piggybacks on something companies already track, as they pay per token for API usage.

Understanding Tokens in AI

In the context of AI, a token is a unit of text that represents the computational complexity of processing language. Tokens can be words, parts of words, or even punctuation marks. The more complex the computation, the more tokens are required. This metric is already widely used in the industry, as companies typically pay for API usage based on the number of tokens processed.

Advantages of the Token Approach

Using tokens to measure carbon footprint has several advantages:

  1. Existing Infrastructure: Companies already track token usage for billing purposes, making it easier to implement this measurement approach.

  2. Scalability: The method can be applied across different models and use cases.

  3. Precision: Tokens provide a more granular measure of computational work than simply tracking overall energy consumption.

Key Variables Affecting Energy Consumption

To accurately measure AI's carbon footprint using the token approach, we need to consider three main categories of variables:

Model-related factors

  • Model size and number of parameters

  • Processing architecture (e.g., transformer-based models)

  • Token context length

  • Context window size

These factors directly influence how many computations are needed to process each token. Larger models with more parameters and longer context windows typically require more energy per token.

Hardware-related factors

  • Hardware efficiency

  • Processing capabilities (batch size and parallelization)

  • GPU specifications

The efficiency of the hardware running the AI model significantly impacts energy consumption. More efficient GPUs and better parallelization can reduce the energy needed per token.

Data center-related factors

  • Power Usage Effectiveness (PUE)

  • Local energy grid characteristics

  • Regional variations in renewable energy mix

The overall efficiency of the data center and the source of its electricity play crucial roles in determining the carbon footprint. A data center powered by renewable energy will have a lower carbon footprint per token than one relying on fossil fuels, even if their energy consumption is the same.

Implementing the Token Approach

To implement this measurement method, organizations would need to:

  • Track token usage across their AI operations

  • Gather data on the relevant variables mentioned above

  • Develop a formula that converts token usage to energy consumption and then to carbon emissions

  • Regularly update their calculations as hardware efficiency improves and energy sources change

Calculating Energy Per Token: A Case Study

​​To illustrate the process of measuring AI's energy consumption and carbon footprint, let's examine a theoretical deployment of GPT-4 on a Google Cloud data center in Korea. This case study will demonstrate how we can apply the token-based approach to estimate the environmental impact of a large language model in real-world conditions.

For this case study, we'll consider a hypothetical scenario where GPT-4 is being used for various natural language processing tasks. We'll make several key assumptions based on publicly available information and industry standards:

  • Model complexity: Approximately 280 billion parameters

  • Context length: 8,000 tokens (standard for GPT-4)

  • Average context window: 1,250 tokens

  • Hardware: NVIDIA A100 GPU, consuming 400 watts and processing 1,400 tokens per second

  • Google Cloud's Power Usage Effectiveness (PUE): 1.1

  • Korea's grid carbon intensity: 0.459 kg CO₂/kWh

These assumptions provide a foundation for our calculations, allowing us to estimate the energy consumption and carbon emissions associated with running GPT-4 in this specific environment.

Step-by-Step Calculations

Let's walk through the process of calculating the energy consumption and carbon emissions per token:

  • We start by calculating the base energy per token for GPT-4, which comes to 0.0004571 kWh/token.

  • Next, we adjust this figure for the average context window, resulting in 0.0001700 kWh/token. This adjustment accounts for the fact that not all tokens in the context window require the same amount of processing.

  • We then factor in the data center's PUE, which gives us 0.000187 kWh/token. This step accounts for the additional energy required for cooling and other data center operations.

  • Converting to Joules, we arrive at a final energy consumption of 673.2 Joules/token.

  • Finally, using Korea's grid carbon intensity, we can estimate the carbon emissions at approximately 0.09 grams CO₂/token.

To put this into perspective, for an average inference of 200 tokens (roughly equivalent to a paragraph of text), the total carbon emissions would be about 1.8 grams of CO₂.

While this might seem small, when scaled to millions of queries per day, the environmental impact becomes significant.

The Path Forward: Building Green AI

As we consider the implications of these calculations, it's worth examining India's approach to AI development. The country's unique power constraints have necessitated innovative thinking about AI implementation, potentially leading the way in developing green AI solutions.

India's approach focuses on three key areas

Use case driven approach: By prioritizing AI applications that solve real-world problems, India ensures that energy consumption is justified by tangible benefits. This approach involves designing AI systems with power constraints in mind from the start and optimizing for both performance and energy efficiency.

Standardization and reporting: India is working towards establishing standardized methods for tracking and reporting energy consumption and emissions throughout the AI lifecycle. This includes setting baseline measurements and targets, and creating consensus on calculation methods and assumptions.

Optimization opportunities: Indian developers are exploring various ways to optimize AI energy consumption, including:

  • Choosing efficient cloud providers and hardware

  • Considering regional energy grid characteristics in deployment decisions

  • Designing for "Green AI" principles

  • Focusing on computational efficiency in model architecture

These strategies, born out of necessity in India, offer valuable insights for global efforts to develop more sustainable AI systems.

Next Steps

While the exact calculations may be debatable, the framework for measuring AI's environmental impact using tokens is feasible.

The next step is forming a consortium to agree on standard assumptions and create templates for consistent measurement and reporting. This standardization will enable the development of more environmentally conscious AI systems and support the growth of green AI initiatives.

As ESG laws tighten globally, having a standardized way to measure and report AI's carbon footprint becomes crucial. By piggy-backing on existing token-based metrics, we can create a practical system for tracking and managing AI's environmental impact.

We invite you to provide feedback and participate in this project.

Write to david@peopleplus.ai and anuragsridharanwork@gmail.com.

Join the Community

People+ai is a non-profit, housed within the EkStep Foundation. Our work is designed around the belief that technology, especially ai, will cause paradigm shifts that can help India & its people reach their potential.