AI just solved a math problem that mathematicians couldn’t. This forms part of a broader trend of rapid capability advances, but the ability to control ever more powerful AIs remains elusive.
What happens when AI solves a "math" problem with a solution that is too complex for humans to comprehend? Should we just trust that the answer is correct?
That's a good question. In the case of mathematics, you can formalize proofs and then use formal proof-checking software to confirm that it checks out.
From what I understand, in practice, formalizing a proof from what you'd see in a paper into something you can feed into a proof-checker can take a lot of time and effort, though.
But in principle, it seems possible to verify proofs that even humans don't understand.
In most other domains I don't think you'd be able to get these sorts of guarantees though.
Is a proof checker a computer program? I might have read and watched too much syfy. I just find it easy to imagine AI providing an incorrect answer to a very complicated question. An incorrect answer that could result in a major calamity for humanity. Like I said, too much syfy.😁
Yeah, it is. I mean, I think the scenario you describe is completely possible - was just pointing out that in math it's at least in principle possible to verify things with traditional software that doesn't misbehave.
But many things are not math. It's easy to imagine realistic scenarios where, for example, a President comes to rely on a very powerful AI for high stakes decisions, like going to war, or an AI CEO trusts the advice of one AI about the risks of deploying a more powerful AI. And if these AIs makes mistakes or actually work to undermine us, that could end really badly.
Yes, proofs can be formalized and automatically checked. But you're actually on to something here.
It's possible to mathematically prove that a computer program will result in a specific output, but then when you actually run it, it does something else. How? Because it performs its mere logical operations in a way that takes advantage of the physics of the hardware it is running on, in order to produce unexpected outcomes. (Search up Row Hammer for an example. You can flip a bit back and forth so fast that it ends up flipping a bit next to it.)
You can prove the mathematical implications of a system, but still be uncertain of its implications in the real world. We can get a provably correct answer to a wrong question. If we are up against an adversary that is more intelligent, that can essentially think in more dimensions than we can, then there is no possible defense against this, because we don't know what we don't know.
What happens when AI solves a "math" problem with a solution that is too complex for humans to comprehend? Should we just trust that the answer is correct?
That's a good question. In the case of mathematics, you can formalize proofs and then use formal proof-checking software to confirm that it checks out.
From what I understand, in practice, formalizing a proof from what you'd see in a paper into something you can feed into a proof-checker can take a lot of time and effort, though.
But in principle, it seems possible to verify proofs that even humans don't understand.
In most other domains I don't think you'd be able to get these sorts of guarantees though.
Is a proof checker a computer program? I might have read and watched too much syfy. I just find it easy to imagine AI providing an incorrect answer to a very complicated question. An incorrect answer that could result in a major calamity for humanity. Like I said, too much syfy.😁
Yeah, it is. I mean, I think the scenario you describe is completely possible - was just pointing out that in math it's at least in principle possible to verify things with traditional software that doesn't misbehave.
But many things are not math. It's easy to imagine realistic scenarios where, for example, a President comes to rely on a very powerful AI for high stakes decisions, like going to war, or an AI CEO trusts the advice of one AI about the risks of deploying a more powerful AI. And if these AIs makes mistakes or actually work to undermine us, that could end really badly.
Yes, proofs can be formalized and automatically checked. But you're actually on to something here.
It's possible to mathematically prove that a computer program will result in a specific output, but then when you actually run it, it does something else. How? Because it performs its mere logical operations in a way that takes advantage of the physics of the hardware it is running on, in order to produce unexpected outcomes. (Search up Row Hammer for an example. You can flip a bit back and forth so fast that it ends up flipping a bit next to it.)
You can prove the mathematical implications of a system, but still be uncertain of its implications in the real world. We can get a provably correct answer to a wrong question. If we are up against an adversary that is more intelligent, that can essentially think in more dimensions than we can, then there is no possible defense against this, because we don't know what we don't know.
That's a good point
IT IS INDEED!
DEFINATELY!
CONTROL AND REGULATE AI AT ONCE!!!! NO MORE DAMN DEEP FAKES!!!!!