Yudkowsky lists some reasons why certain highly intelligent people are only smart at being stupid:
- You become overly skilled at defending beliefs you arrived at for unskilledreasons;
- Success on comparatively easy problems in the past, leads to overconfidence on more difficult problems in the future;
- Because most of the advice offered you comes from people (apparently) not as smart as you, you become intellectually isolated;
- You spend too much time as the smartest person in the room and can’t handle it emotionally;
- Because you enjoy looking smart, you avoid trying new and difficult things where you might not perform as well, and so become very narrow;
- Because you enjoy looking smart, you don’t like confessing your mistakes or realizing your losses;
- Because people praise you for being good at math or something, you assume you are already wise, and you don’t apply your intelligence to becoming wiser in other areas.
Yudkowsky ridicules behaviorism:
Behaviorism was the doctrine that it was unscientific for a psychologist to ascribe emotions, beliefs, thoughts, to a human being. After all, you can’t directly observe anger or an intention to hit someone. You can only observe the punch. You may hear someone say “I’m angry!” but that’s hearing a verbal behavior, not seeing anger. Thoughts are not observable, therefore they are unscientific, therefore they do not exist. Oh, you think you’re thinking, but that’s just a delusion – or it would be, if there were such things as delusions.
Behaviorists (at least the radical ones) postulate that the organism/mind doesn’t matter. But that can’t be true since cats can’t learn calculus, in fact many humans can’t learn calculus!
Yudkowsky is not a logical positivist, although concepts like “beliefs should pay rent in anticipated experiences” are very similar to positivist credos. He thinks of
logical positivism as another case in point of “mistaking the surface of rationality for its substance”.
Consider the hypothesis:
On August 1st 2008 at midnight Greenwich time, a one-foot sphere of chocolate cake spontaneously formed in the center of the Sun; and then, in the natural course of events, this Boltzmann Cake almost instantly dissolved.
I would say that this hypothesis is meaningful and almost certainly false. Not that it is “meaningless”. Even though I cannot think of any possible experimental test that would discriminate between its being true, and its being false.
Not every untestable assertion is false; a deductive consequence of general statements of high probability must itself have probability at least as high. So I do not believe a spaceship blips out of existence when it crosses the cosmological horizon of our expanding universe, even though the spaceship’s existence has no further experimental consequences for me.
If logical positivism / verificationism were true, then the assertion of the spaceship’s continued existence would be necessarily meaningless, because it has no experimental consequences distinct from its nonexistence. I don’t see how this is compatible with a correspondence theory of truth.
To be unable to produce an experiential distinction from a belief, is usually a bad sign – but it does not always prove that the belief is meaningless. A great many untestable beliefs are not meaningless; they are meaningful, just almost certainly false…
Some commenters point out that he attacks a straw-man-version of logical positivism, e.g. komponisto:
“Eliezer, you’re definitely setting up a straw man here. Of course it’s not just you — pretty much everybody suffers from this particular misunderstanding of logical positivism.
“Untestable” does not mean “untestable by humans using current technology”. What it means is untestable, period — even by (say) a hypothetical being with godlike powers. This is what distinguishes a chocolate cake in the sun from “post-colonial alienation”. If a chocolate cake spontaneously formed in the Sun, there would be physical consequences. These consequences would necessarily be detectable to sufficiently advanced beings. We need only imagine a Laplacian calculator, for example.
The feasibility of performing the verification is utterly beside the point, because we’re only interested in the meaning of the statement.
Contrast this with “post-colonial alienation”. The problem there is not that we lack some technological gadget. Rather, the problem is that no being, not even a god or a Laplacian demon, could verify the statement. In the case of a chocolate cake, you simply present God with a list of ingredients, and tell him/her to look for them in the Sun; but what do you tell God to look for in the works of Shakespeare in order to determine whether there is “post-colonial alienation”?”