There would still be disagreements on how to program the AI. A super intelligent AI with information about everything could probably find the best way to reach any goal, but how do we define what are our goals in the first place, and how they’re prioritized? And what constraints they are on the actions of the AI state?
There’s a quote I’ve heard, I think it’s from Alexandre Grothendiek: " There’s no systematic way to go from the knowledge of what it to the knowledge of what should be", that sums it up pretty well.
There would still be disagreements on how to program the AI. A super intelligent AI with information about everything could probably find the best way to reach any goal, but how do we define what are our goals in the first place, and how they’re prioritized? And what constraints they are on the actions of the AI state?
There’s a quote I’ve heard, I think it’s from Alexandre Grothendiek: " There’s no systematic way to go from the knowledge of what it to the knowledge of what should be", that sums it up pretty well.