For most of us in the tech industry, the transformative changes of LLMs and generative AI are quite apparent. However, what might be hidden for many is the physical infrastructure powering this LLM-based transformation. Living in Ashburn, Virginia though, I am right at the heart of one of the largest data center hubs in the USA. Although many residents in the area might be unhappy with the proliferation of data centers, the demand for them is growing significantly, and LLMs are a significant driver of that growth.

According to some latest figures, data centers account for approximately 3% of total power consumption in the US. That number is expected to double in next couple of years. The next generation of data center designs are supposed to be larger and more power hungry than existing data centers. It is interesting to learn that the pace at which electrical grid connectivity reaches a new data center is slower than the rate at which a new data center comes online. This means that companies are planning for either independent power supplies or moving to areas that are closer to power plants. One example mentioned by Morgan Stanley is that data center operators are planning to build new hubs closer to existing nuclear power plants in order to get access to power more easily.

The big tech companies have all made public commitments to reducing their carbon footprints, and building more efficient data centers. e.g Google, Apple 2030 plan. The need for geographical separation of data, often dictated by local laws, and coupled with the need for redundancy and low latency, the number of data center constructions are increasing across the world. What remains to be seen is whether the renewable energy sources and other power efficiency techniques in data centers can catch up to the ever growing data set of text, audio, video that needs to be constantly indexed and fed into the models. The next time you chat with your favorite LLM, remember it might be powered by a data center next to a nuclear plant.