Artificial Intelligence Theory
A recent AIChE Journal Highlight caught my attention. It is dated March 2025 and has the title ‘Developing a Thermodynamics-Like Theory of Large Language Models’. It is based on analysis by Venkatasubramanian in which he suggests that we will witness a transformation from Large Language Models (LLMs) such as ChatGPT to Large Knowledge Models (LKMs).
This transition will be based on conceptual breakthroughs that advance our understanding of LLMs. He compares this transition to the invention of steam engines, which led to the birth of thermodynamics and statistical mechanics. LLMs are the Newcomen/Watt steam engines of the automated cognition revolution.
The go-to quotation from the article is,
Machine learning innovations such as the backpropagation algorithm, reinforcement learning, deep neural networks, transformers, etc., are like the innovations of Thomas Newcomen, James Watt, and others who invented the steam engine . . . The innovators who developed steam engine technology were not the same people who developed its science.
The author notes that it was the industrial innovators who became rich, not the scientists who developed the principles of thermodynamics.
It is also worth considering the role of geopolitical rivalry. One of the first people to develop thermodynamic theory was Nicolas Léonard Sadi Carnot (1796-1832), a junior officer in the post-Napoleonic French army. He recognized that the recent French defeats were partly due to the fact that the British used superior technology. Therefore, if the French were to keep up, they needed to understand thermodynamics.
If this analogy between thermodynamics and AI is correct, then we can expect to see three things:
The development of AI theory has just started.
The AI theorists will not become as wealthy as the current entrepreneurs.
Geopolitical rivalry will be a driving force behind the development of AI theory.
