October 6 was a watershed moment in mathematical history. OpenAI announced that its most advanced models solved 722 open mathematics problems in 372 areas. Among those results, it made progress on three more Millennium Prize problems, the Hodge Conjecture, the Birch-Swinnerton-Dyer conjecture, and the Riemann Hypothesis. OpenAI’s previous solution of the Navier Stokes problem required 10,000 agents working in concert. Amazingly, a single unreleased model spent an average of about three hours a piece on these problems. The average consumer may soon may be able to harness this intellectual power at very low cost.
That a computer model could produce results of this mathematical importance is stunning. Mathematicians are in a state of shock right now, wondering if they will have jobs in the near future. If mathematicians are about to get pink slips, it might be tempting to conclude that science is close to being solved as well, with all the benefits. The good news—at least for them—is that mathematicians will not lose their jobs, but their jobs will be different. The bad news is that science is far from solved.
To see why science is still not solved, we’ll need to get into what the AI models accomplished in more detail to understand what has and has not been accomplished. I’ll review two new OpenAI results that are arguably the best evidence of the arrival of mathematical super intelligence, the progress on the Riemann hypothesis, and the resolution of the Unique Games Conjecture. I’ll also return to the Navier Stokes problem that I covered in a previous post.
By reviewing what OpenAI accomplished, we’ll see that a model’s ability to derive the logical implications of axioms and theorems—the essence of pure mathematics—is virtually unlimited. But science requires us to make non-trivial, true, and predictive statements about the real world. Logical consistency is not enough. Science also requires confrontation of theories with experiment and observation, a messy, expensive, and brittle endeavor. Superhuman mathematical ability has removed one bottleneck to scientific progress, but that bottleneck is not the most important one.
Progress on the Riemann Hypothesis
The progress made on the Riemann Hypothesis was probably the most important and impressive achievement among the 772 results. The Riemann Hypothesis, the most important open question in analytic number theory, is about the distribution of prime numbers. Since Euclid, we know that there are an infinite number of primes and we have learned that they spread out farther and farther from each other as we go out to infinity. But can we bound how far they spread out more precisely? That’s what we can do if the Riemann Hypothesis turns out to be true.
Suppose you are walking along a line of the infinite numbers. When you first start walking, you encounter lots of prime numbers. As you keep walking towards greater numbers, you encounter them less and less frequently. If you go way out into the vast space of numbers and stop at an unimaginably high number, where are the primes? As you get higher and higher on the number line, the primes become more and more sparse. How far would you have to walk in the direction of larger numbers to find the next prime? How far would you have to turn around to find a prime?
The Riemann Hypothesis is like a lantern that allows you to illuminate the way forwards and backwards as you walk along the infinite number line, showing you where the next prime numbers are likely to be.
Suppose you have been wandering on the number line, not paying attention, and then stop somewhere and asked yourself, “how many primes are behind me?” Mathematicians know how to make a simple prediction on the number of primes behind you. But how big is the error in that prediction? The Riemann hypothesis allows us to bound the error of that prediction. If the Riemann Hypothesis is true, the ability to bound the error in that prediction also reveals deep structure in the prime numbers. OpenAI didn’t solve the Riemann hypothesis, but it did show how to bound the error of the prediction of primes much better than we can using currently known methods. It did so by making progress on the central question in the Riemann hypothesis.
Without knowing whether the Riemann hypothesis is true, mathematicians have been able to bound the error in the prime number prediction using the so-called classical bound. The progress OpenAI made allows a much better bound.
I’ve illustrated the progress in the table below. In the first column, I have progressively larger and larger numbers. Think of each row in that column as a number that you stop on as you take a walk on the infinite number line. The second column has the central prediction of the number of primes behind you. The third column has an illustrative, order of magnitude classical bound on the error in the prediction. For technical reasons, these bounds are not precise estimates, but rather order of magnitude projections that work asymptotically, i.e., as the numbers get larger and larger. The fourth column shows the OpenAI-implied new order of magnitude bound for the error in the central prediction of the number of primes behind you
You might ask, why go to all this trouble to create this table? I did it to illustrate the subtlety in the OpenAI advance. Notice that when x is below the number 1 with fifteen zeros after it, colored in yellow, the classical bound is actual smaller than the new bound. In fact, when x is small, the new bound can be larger than x itself! After about 1e15, the new bound becomes smaller relative to the classical bound. The new bound is not necessarily useful for making large number predictions about the distribution of primes for large, but finite, numbers. Bounds are an asymptotic statement. They are useful for theoretical math, but they don’t necessarily help for applied math.
The advance in the Riemann hypothesis made by OpenAI’s AI models is a staggering achievement for pure mathematics. But it’s not really useful in practice. It won’t build better cars. It won’t produce fusion energy. It won’t move the level of GDP or its growth rate one iota.
Resolution of the Unique Games Conjecture
The Unique Games Conjecture is one of the most important problems in theoretical computer science. It was formulated in 2002 by mathematician and computer scientist Subhash Khot, who is at the Courant Institute at NYU. The conjecture concerns the following problem: if we somehow knew that an almost perfect solution to a problem exists, how hard is it to find an approximate correct solution? Unfortunately, it’s hard. That’s the implication of OpenAI’s proof that the conjecture is true.
To understand the problem behind the Unique Games Conjecture, consider a simple problem. Suppose we have three nodes, A, B, and C. Each node can be assigned a 0 or 1. We also have three constraints:
A and B must have the same value
B and C must have the same value
A and C must have different values
Can we satisfy all three constraints? No, we can only satisfy two of the constraints, because if A = B and B = C, A ≠ C. In this case, we can satisfy two-thirds of the constraints.
Now, generalize this problems to having millions of nodes, millions of constraints, and perhaps thousands of possible numbers that can be assigned to each node. Suppose we know that 99.9% of the constraints can be satisfied somehow. How hard is it to find a solution in which only 1% of the constraints are satisfied?
Intuitively, it might seem it should be much easier to find the solution to the 1% case than it would be to find the solution to the 99.9% case. The Unique Games Conjecture says both problems are equally hard. The technical term is that the problems are NP-hard. We know now the conjecture is true, assuming OpenAI’s proof survives verification.
The conjecture is important in theoretical computer science because many optimization problems can be mapped into unique games. The conjecture implies that the theoretical solutions we have to many optimization problems are essentially the best possible.
How much does it matter practically?
It’s important to understand that this is a theoretical result. In the world of engineering and applied mathematics, people often simplify problems so that they remain relevant but are practically solvable. The problem can be changed so that it has a better practical solution. We might bring additional information to bear, we might exploit the special structure of the problem, or we might reduce the size of the problem. Moreover, the Unique Games Conjecture doesn’t even apply to all optimization problems that occur practically in industry. As in the case of the Riemann Hypothesis, the advance that OpenAI made is more about pure rather than applied mathematics.
The Navier-Stokes Solution
I covered the Navier-Stokes solution in a previous post. However, I didn’t discuss the practical implications. OpenAI’s AI model’s solution is a monumental achievement for pure mathematics, but it has no relevance to the real world. As a reminder, the Navier-Stokes problem, one of the Millennium Problems, asked whether a smooth, finite force can be applied to a fluid in such a way that the fluid can be caused to achieve an infinite velocity in a finite time. The answer is yes, you can do that.
Does it matter? No, not at all. The question is a mathematical curiosity in the theory of non-linear partial differential equations. In the real world, we know that the speed of light is the ultimate speed limit in the universe. There is no such thing as an infinite velocity. Besides, once the speed of anything gets close to the speed of light, the physics behind the Navier-Stokes equations would cease to apply. We need to bring in relativity.
The Navier-Stokes solution also requires the fluid to get infinitely thin as its velocity increases without bound. That can’t happen either in the real world. Once we get to the atomic level, the physics behind the Navier-Stokes equations would no longer apply. We need to bring in quantum mechanics.
In the world of science and engineering, the Navier-Stokes equations are incredibly important. For real world applications they are solved numerically over the ranges in which they are physically relevant. The most important unsolved practical problem in Navier-Stokes is how to handle fluid turbulence. OpenAI’s solution does nothing to resolve that question.
Science is Different From Pure Mathematics
In pure mathematics, you need to derive the logical implications from a set of definitions, axioms, and theorems. The truth or falsity of the derivations are objective, depend only on the logical rules, and do not rely on external observations of the real world. Neural networks can be trained to perform pure math tasks at a superhuman level, just as they can do so for games such as Chess or Go. In these games, you have to follow a set of rules, the legality of each move can be objectively determined, and its clear who won and who lost. You don’t need to perform experiments to make your next chess move or to prove the Riemann hypothesis.
Science is fundamentally different from pure mathematics. In science, all that counts is agreement with observation and correct empirical predictions. We are no longer in the world of pure reason. As physicist Richard Feynman pointed out, knowing non-trivial true facts about the outside world of experience is incredibly hard. It requires diligence, indefatigable work, and messy, expensive experimentation.
For example, in practical applications of Navier-Stokes, applied mathematicians solve the problem numerically, simplifying if possible, to make it tractable. Once there is a solution, however, it must be checked against observations. If the application is the design of a jet engine, the solution must be checked in a laboratory environment that simulates jet engine combustion. If the fluid application is a plasma in a nuclear fusion reactor, the solution must be checked in realistic fusion conditions. All this is expensive, hard, and time-consuming.
A Further Bottleneck To Moving the GDP Needle: Cost
Even if a genuine advancement is made in a scientific field, it does not follow that living standards will improve. The solution must also be cost-effective. We need entrepreneurs to solve that problem.
The ancient Greeks could build a rudimentary steam engine and yet there were no steam engines for thousands of years. Many people invented the light bulb, but it was Edison who figured out how to take the technical solutions available and make one economically practical. That’s why lighting is ubiquitous and cheap.
The book Slide Rule by Nevil Shute chronicles his struggle as one of the first aeronautical engineers and entrepreneurs to make flight economically practical. In the 1920s, fixed wing flight was deemed too expensive and so all of the work focused on dirigibles. Eventually, entrepreneurs like Shute discovered how to make fixed wing flight much cheaper and airline travel is now commonplace. (Shute went on to become a celebrated author of novels such as “On the Beach,” which was made into a feature film in 1959.)
We have known technically how to get into space for almost 75 years now. But space has never been economically practical, until recently. The reason SpaceX has devoted so much time to making re-usable rockets work is that rockets are much too expensive to throw away after one use if we expect to put goods and services in orbit at costs people can afford. SpaceX has been working on reducing the cost of putting a kilogram into orbit by a factor of ten, and then by another factor of ten.
The history of successful technology is the record of how entrepreneurs took basic science and engineering and got the costs down. Science will not affect GDP without scientists who perform experiments and entrepreneurs who reduce the costs. Mathematics is the language of some scientific fields and super human mathematical intelligence removes one bottleneck, at least in some areas of science. But the other, very difficult bottlenecks to economic progress remain.
Why Is OpenAI Solving These Math Problems Then? Marketing
Who is asking OpenAI to solve these open mathematics problems? Is industry clamoring for a resolution of the Millennium problems? No, not really. Could it be the mathematicians who are asking for them? It’s certainly not them. Mathematicians are in a general freakout over the threat to their jobs. The advisory committee of prominent mathematicians that OpenAI appointed to advise them issued a clear recommendation, in effect “Keep your grubby hands off our problems.” OpenAI is not listening to the committee. The so-called Association of Human Mathematicians said the same thing
Mathematician and computer scientist Scott Aaronson is in a state of despair. In his blog post following the October 6th announcement, he noted his 9-year old son taunted his mother because OpenAI just nuked the Unique Games Conjecture, the problem she based her career on.
If no one is asking for these pure math solutions, why is OpenAI producing them, especially when the math professors are so upset about it? It’s marketing. As I’ve discussed before, the Olympian valuations of the frontier model companies can’t be justified unless you believe AGI is around the corner or you expect a cornucopia of new scientific discoveries that cause the economic growth rate to double. Solving famous open mathematics problems is a way to keep the buzz going, even if they don’t matter in the real world.
The new scientific wonders are not coming though unless the focus is changed to real technology problems that people will pay dearly to get solved. The models are now astonishingly good. For me, they increase my productivity by a factor of ten. But I’m not building cars; I’m not designing fusion reactors; I’m not working on cancer cures. Those pursuits require real world experimentation and testing. They require disciplined management to reduce costs. Superhuman mathematical AI doesn’t help with any of that.
Unless things change, the valuations of the frontier model companies are too high. Even though I’m stunned by the quality of their latest models, I wouldn’t buy Anthropic IPO stock except at a significant discount to the valuations people have been touting.
Appendix:
Last year, I put together a tutorial on the Riemann Hypothesis that attempts to explain what it is. My earlier tutorial was designed to take you deeper than the many popularizations do, while using only fairly elementary mathematics. For those who are interested in more details on the OpenAI Riemann hypothesis progress, read on.
Mathematicians have known that the critical zeros of the zeta function lie in the strip between 0 and 1. The Riemann hypothesis says all the critical zeros lie on the line with real part 1/2.
Mathematicians know we can approximate the number of primes less than some number x as
where 𝛑(x) is the prime counting function, the number of primes less than or equal to x. From the formula we see the number of primes less than or equal to x can be approximated by the Li(x) function
plus an error term that gives an order of magnitude of the size of the error.
The OpenAI result showed that the critical zeros must lie in the strip between 1/8 and 7/8, allowing us to bound the error in the approximation of the prime counting function as follows
With the new error bound, the order of magnitude of the error eventually becomes smaller than the old error bound as x becomes larger and larger. If the Riemann Hypothesis were proved to be correct, the error bound could be sharpened even further to
The table above compares the classical bound to the new OpenAI 7/8ths bound. Just for exposition, I used c = 0.1 in the table. But the value of c doesn’t matter. Asymptotically, the classical error bound will be eventually be larger than the new error bound.