Rather unimportant posts.
Building machines that are more intelligent than you are is not paradoxical. It’s just hard.
Deep Blue played chess barely better than the world’s top humans, but a heck of a lot better than its own programmers. Deep Blue’s programmers had to know the rules of chess – since Deep Blue wasn’t enough of a general AI to learn the rules by observation – but the programmers didn’t play chess anywhere near as well as Kasparov, let alone Deep Blue.
…The programmers built a better move generator – one that more powerfully steered the game toward the target of winning game positions. Deep Blue’s programmers surely had some slight ability to find chess moves that aimed at this same target, but their steering ability was much weaker than Deep Blue’s.
It is futile to protest that this is “paradoxical”, since it actually happened.
Complexity has little to do with intelligence, especially if you take Kolmogorov complexity. After all, random noise has the highest Kolmogorov complexity.
At a Foresight Gathering some years ago, a Congressman was in attendance, and he spoke to us and said the following:
“Everyone in this room who’s signed up for cryonics, raise your hand.”
Many hands went up.
“Now everyone who knows the name of your representative in the House, raise your hand.”
Fewer hands went up.
“And you wonder why you don’t have any political influence.”
Rationalists would likewise do well to keep this lesson in mind.