Nothing new in this post, although it’s a very good and clear summary:
I fear that in my drive for full explanation, I may have obscured the punchline from my theory of metaethics. Here then is an attempted rephrase:
There is no pure ghostly essence of goodness apart from things like truth, happiness and sentient life.
What do you value? At a guess, you value the life of your friends and your family and your Significant Other and yourself, all in different ways. You would probably say that you value human life in general, and I would take your word for it, though Robin Hanson might ask how you’ve acted on this supposed preference. If you’re reading this blog you probably attach some value to truth for the sake of truth. If you’ve ever learned to play a musical instrument, or paint a picture, or if you’ve ever solved a math problem for the fun of it, then you probably attach real value to good art. You value your freedom, the control that you possess over your own life; and if you’ve ever really helped someone you probably enjoyed it. You might not think of playing a video game as a great sacrifice of dutiful morality, but I for one would not wish to see the joy of complex challenge perish from the universe. You may not think of telling jokes as a matter of interpersonal morality, but I would consider the human sense of humor as part of the gift we give to tomorrow.
And you value many more things than these.
In short: What we find funny, beautiful, joyful, true, elegant, fair, etc. is right. How does this differ from mere preferences?
Most goods don’t depend justificationally on your state of mind, even though that very judgment is implemented computationally by your state of mind. A personal preference depends justificationally on your state of mind.
If somebody altered my brain so that I prefered green tea to black tea, it would be right for me to drink green tea. But if my brain were altered so that I prefered raping women to normal sex, raping women would still be wrong.
And there’s no one value that determines whether a complicated event is good or not—and no five values, either. No matter what rule you try to describe, there’s always something left over, some counterexample. Since no single value defines goodness, this can make it seem like all of them together couldn’t define goodness. But when you add them up all together, there is nothing else left.
Again, values are fragile and complex.
When you ask “But which utility function should I use?” the word should is something inseparable from the dynamic that labels a choice “should”—inseparable from the reasons like “Because I can save more lives that way.”
Every time you say should, it includes an implicit criterion of choice; there is no should-ness that can be abstracted away from any criterion.
I sometimes say that morality is “created already in motion“.
circular reasoning recursive justification. All criteria you apply when you’re reasoning about your ethical beliefs are themselves grounded in and justified by other criteria of your ethics.
There is no perfect argument that persuades the ideal philosopher of perfect emptiness to attach a perfectly abstract label of ‘good’. The notion of the perfectly abstract label is incoherent, which is why people chase it round and round in circles. What would distinguish a perfectly empty label of ‘good’ from a perfectly empty label of ‘bad’? How would you tell which was which?
Humans have a cognitive architecture that easily misleads us into conceiving of goodness as something that can be detached from any criterion.
…Very sad. I too was hoping for a perfectly abstract argument; it appealed to my universalizing instinct. But…
But the question then becomes: is that little fillip of human psychology, more important than everything else? Is it more important than the happiness of your family, your friends, your mate, your extended tribe, and yourself? If your universalizing instinct is frustrated, is that worth abandoning life? If you represented rightness wrongly, do pictures stop being beautiful and maths stop being elegant? Is that one tiny mistake worth forsaking the gift we could give to tomorrow? Is it even really worth all that much in the way of existential angst?
Or will you just say “Oops” and go back to life, to truth, fun, art, freedom, challenge, humor, moral arguments, and all those other things that in their sum and in their reflective trajectory, are the entire and only meaning of the word ‘right’?
Here is the strange habit of thought I mean to convey: Don’t look to some surprisingunusual twist of logic for your justification. Look to the living child, successfully dragged off the train tracks. There you will find your justification. What ever should be more important than that?
If we cannot take joy in things that are merely good, our lives shall be empty indeed.
But then why is Yudkowsky such an ardent proponent of utilitarianism? I’ve always held utilitarianism in high regard (these days I’m more ambiguous about it) because it seemed like a reasonable and objective moral philosophy. But I don’t see why utilitarianism follows from Yudkowsky’s meta-ethics. OTOH I don’t know any consistent moral philosophy that follows from Yudkowsky’s meta-ethics.
Anyway, here’s an insightful comment by Yudkowsky which shows that he has at least some sympathies for anti-natalism:
“My position on natalism is as follows: If you can’t create a child from scratch, you’re not old enough to have a baby.
This rule may be modified under extreme and unusual circumstances, such as the need to carry on the species in the pre-Singularity era, but I see no reason to violate it under normal conditions.”
Yeah, not procreating before the singularity is a no-brainer.