In the book “Permutation City” by Greg Egan, apparently the favorite Sci-Fi book of Yudkowsky one of the main characters, Peer, modifies himself to find table-leg-carving utterly fascinating and enjoyable. Yudkowsky is horrified by this vision and thinks that…
…at that point, you might as well modify yourself to get pleasure from playing Tic-Tac-Toe, or lie motionless on a pillow as a limbless eyeless blob having fantastic orgasms. It’s not a worthy use of a human-level intelligence.
Why did Peer choose to do this? Because he already solved the mystery of consciousness, all mathematical riddles, wrote operas, etc. – in short he exhausted the enterity of Fun Space.
Now, I agree with Yudkowsky that this is unlikely. Fun Space is probably very, very large. Furthermore, the smarter you are, the more problems you can tackle, the larger Fun Space becomes. Just think about which problems Chimpanzees can enjoy and what you can enjoy. Human Fun Space is orders of magnitude larger than Chimpanzee Fun Space.
Heck, most humans can only have fun through drugs, TV, sport, music or sex. They don’t really read books. They don’t think about philosophical questions. They can’t enjoy complex movies. Fun Space_IQ-145-human is vastly bigger than Fun Space_senile grandpa.
OTOH, if you’re smarter you can’t enjoy trivial things anymore. And you can learn faster and…
Learning is fun, but it uses up fun: you can’t have the same stroke of genius twice. Insight is insight because it makes future problems less difficult, and “deep” because it applies to many such problems.
If you get smarter over time (larger brains, improved mind designs) that’s a still higher octave of the same phenomenon. (As best I can grasp the Law, there are insights you can’t understand at all without having a brain of sufficient size and sufficient design. Humans are not maximal in this sense, and I don’t think there should be any maximum—but that’s a rather deep topic, which I shall not explore further in this blog post. Note that Greg Egan seems to explicitly believe the reverse—that humans can understand anything understandable—which explains a lot.)
As I mentioned, I really hope there is some, due to our limited intellect currently incomprehensible explanation which shows that there is something deeply meaningful about our universe and that our existence has some higher purpose than merely having the maximum amount of fun possible. Maybe it has something to do with consciousness or morality or so, and because consciousness and morality are mysterious and the meaning of life is mysterious they obviously are related or something like that…sigh.
Anyway, Yudkowsky offers further arguments that try to show that we’ll never run out of fun:
…Fun Space can increase much more slowly than the space of representable problems, and still overwhelmingly swamp the amount of time you could bear to spend as a mind of a fixed level. Even if Fun Space grows at some ridiculously tiny rate like N-squared—bearing in mind that the actual raw space of representable problems goes as 2N—we’re still talking about “way more fun than you can handle”.
But even on the arguments given so far… I don’t call it conclusive, but it seems like sufficient reason to hope and expect that our descendants and future selves won’t exhaust Fun Space to the point that there is literally nothing left to do but carve the 162,329th table leg.
Here is a great counter-argument by Wei Dai:
“One day we’ll discover the means to quickly communicate insights from one individual to another, say by directly copying and integrating the relevant neural circuitry. Then, in order for an insight to be Fun, it will have to be novel to transhumanity, not just the person learning or discovering it. Learning something the fast efficient way will not be Fun because there’s not true effort. Pretending that the new way doesn’t exist, and learning the old-fashioned way, will not be Fun because there’s not true victory.
I’m not sure there are enough natural problems in the universe to supply the whole of transhumanity with an adequate quantity of potential insights. “Natural” meaning not invented for the sole purpose of providing an artificial challenge. Personally, I can’t see how solving the n-th random irrelevant mathematical problem is any better than lathing the n-th table leg.”
Our senses are kinda dumb. They are better suited for e.g. eating hamburgers than reading or philosophizing.
Writing a complicated computer program carries its own triumphs and failures, heights of exultation and pits of despair. But is it the same sort of sensual experience as, say, riding a motorcycle? I’ve never actually ridden a motorcycle, but I expect not.
I’ve experienced the exhilaration of getting a program right on the dozenth try after finally spotting the problem. I doubt a random moment of a motorcycle ride actually feels better than that. But still, my hunter-gatherer ancestors never wrote computer programs.
Yeah, it would be really awesome if we got comparable adrenaline rushes from reading a book and running away from a tiger.
…I do want to nudge people into adopting something of a questioning attitude toward the senses we have now, rather than assuming that the existing senses are The Way Things Have Been And Will Always Be. A sex organ bears thousands of densely packed nerves for signal strength, but that signal—however strong—isn’t as complicated as the sensations sent out by taste buds. Is that really appropriate for one of the most interesting parts of human existence? That even a novice chef can create a wider variety of taste sensations for your tongue, than—well, I’d better stop there. But from a fun-theoretic standpoint, the existing setup is wildly unbalanced in a lot of ways. It wasn’t designed for the sake of eudaimonia.
Very insightful comment by Nick Tarleton:
“I feel like practice has allowed me to develop a modality for code (really, for informational relationships, control flow, and the like, which includes other things like Bayes-structure) which allows programming to be mildly sensual, and the richness of the aesthetic terms used by hackers makes me think this must be fairly common. Still, of course, not only is the sensation much weaker than anything I have natural wetware for, but the lack of a direct interface between the modality and the actual code makes the experience more like reading a good description of driving a motorcycle than driving a motorcycle.
Speculatively, maybe nerdiness involves a high ability to turn new things into sensual experiences. Jumping off from the point about sex, this could help explain the apparent higher frequency of kinkiness among nerds.”
Nice comment by Yudkowsky:
“…the sad fact of the matter is that Cosmopolitan has to offer 73 different bits of sex advice every month, and long-term couples have to go to such lengths to prevent sexual boredom, just because creating sexual variety is so much more difficult than sprinkling cinnamon on an apple. If our taste buds were as complexity-impoverished as our sex organs, restaurants would also have to drip hot wax on your tongue just to keep you interested.”
Argues that we lose something by not doing things ourselves. It makes more fun to discover a proof of the Pythagorean Theorem yourself than to merely read about one in a book. And you’ll remember it better this way.
Barry Schwartz’s The Paradox of Choice—which I haven’t read, though I’ve read some of the research behind it—talks about how offering people more choices can make them less happy.
…the pain of losing something is between 2 and 2.5 times as worse as the joy of gaining it….
So—if you can only choose one dessert, you’re likely to be happier choosing from a menu of two than a menu of fourteen. In the first case, you eat one dessert and pass up one dessert; in the latter case, you eat one dessert and pass up thirteen desserts. And we are more sensitive to loss than to gain.
…Suppose your computer games, in addition to the long difficult path to your level’s goal, also had little side-paths that you could use—directly in the game, as corridors—that would bypass all the enemies and take you straight to the goal, offering along the way all the items and experience that you could have gotten the hard way. And this corridor is always visible, out of the corner of your eye.
Even if you resolutely refused to take the easy path through the game, knowing that it would cheat you of the very experience that you paid money in order to buy—wouldn’t that always-visible corridor, make the game that much less fun?
Yeah, walk-throughs certainly ruin adventure games.
10. Devil’s Offers
Argues that a superintelligent AI that goes around and grants every wish is suboptimal. Firstly, we often don’t know what we want and secondly, as mentioned in the previous post, the mere option of having the easy solution of just asking the superintelligent AI to solve your problem makes you less happy.