Probability estimates: Metaethics

[Epistemic note: This is an old post and probably doesn’t accurately reflect my views anymore. Although it could be possible.]


This post summarizes my metaethical views and my confidence in their correctness. I assign probabilities to every relevant metaethical category. (Beware: My classification-system may be quite idiosyncratic.) I also add a short description, so that you know what I mean by the respective term. Thus we don’t have to argue about mere definitions.

Initially, I wanted to outline the normative implications of every discussed metaethical theory. But I realized that my classification-scheme is way to coarse-grained. Even if I assume that e.g. absolute moral realism is right, I still don’t have a clue which specific normative moral theory is right and those normative theories have widely divergent implications. And even if I knew that a specific normative theory like e.g. preference-utilitarianism were true, I still wouldn’t know what to do! Prima facie promising strategies like e.g. contributing to FAI-research could have catastrophic consequences whereas furthering WBE-research could be the morally optimal strategy, but these sorts of questions have little to do with metaethics per se. Which shows that all my work was in vain. I hate this place.

And obviously, this is all subjective and quite volatile since I’m used to revise my philosophical worldview fairly regularly.

1. Cognitivism vs. Non-Cognitivism

This distinction is mainly about the semantics of moral discourse and not that important but I included it for completeness’ sake. There’s also great confusion about the definitions of those terms. Some Non-Cognitivists claim that moral judgments have a descriptive component but are mainly emotive. This comes awfully close to Cognitivism, if you ask me.

P( Cognitivism) + P (Non-Cognitivism) = 1

1.1 Cognitivsm: 80%

Description: Moral judgments express beliefs, are therefore truth-apt, i.e. can be true or false.

1.2 Non-Cognitivism: 20%

Description: Moral judgments are mere expressions of emotions or norms.

2. Moral Ontology

Are there (mind-independent) moral facts or not?

P(Weak Moral Realism) + P(Strong Moral Realism) + P(Absolute Moral Realism) + P(Moral Anti-Realism) = 1

2.1 Weak Moral Realism (25%)

Description: Moral facts exist, but are not really mind-independent. Yudkowsky’s metaethics belongs to this category. Although the huge computation that is our morality is mind-independent, we wouldn’t care about this computation if we had only somewhat different brains.

In short: Yudkowsky is right. Most humans agree on what is right (at least to a large extent), if they are sufficiently well informed, intelligent and rational. Only few alien species would agree on/be motivated by what is right.

2.2 Strong Moral Realism (20%)

Description: Moral facts exist and are somewhat mind-independent. Most sentient beings that evolved by means of natural selection agree, at least to some degree, on what is right and what is wrong and would be motivated by such considerations. Some of the reasons why this may be the case: Evolutionary adaptive strategies which happen to be almost universal like e.g. kin-selection and empathy could develop into something like altruism and notions like equality/moral supervenience/universality and so on. Almost all “natural” creatures would have the desire to reduce suffering and promote happiness which could be the basis of further moral intuitions and values, etc.

But it would be possible to build a superintelligent, yet unfriendly AI like a paperclipper since a paperclipper is made from scratch.

In short: Most sentient beings agree (at least to some degree) on what is right and are motivated to act accordingly, if they evolved through natural selection and are sufficiently well-informed, intelligent and rational.

2.3 Absolute Moral Realism (20%)

Description: Moral principles are somehow embedded into the very fabric of the universe. One example would be Robert Wright’s theory about non-zero-sum games and the continuing trend of the universe to become ever more complex and cooperative (if I understand him correctly). Other examples are Goertzel’s proposal of the 3 moral principles Joy, Growth and Choice or Plato’s Theory of Forms. I guess Kant’s categorical imperative is another one.

In short: All sentient beings agree on what is right, if they are sufficiently well informed, intelligent and rational.

2.4 Theistic Moral Realism (5%)

Definition: Moral facts exist and are mind-independent, at least with regard to non-divine beings. Something is good and right, because it is in God’s will or the Creator of this universe somehow ensured that there is something objective about moral values. Sentient beings that behave too badly would get punished somehow. 2.4 is a subset of 2.3.

In short: All sentient beings agree on what is right, if they are sufficiently well informed, intelligent and rational.

2.5 Moral Anti-Realism (35%)

Description: There are no moral facts. A prime example would be Mackie’s Error-Theory. I guess Yudkowsky’s metaethics would be anti-realistic if the moral intuitions/values of humans varied to such an extent that many morality-computations would have to be postulated. Yudkowsky’s theory only makes sense, if there exists only one “morality-computation” and not numerous sets of mere preferences that vary greatly from person to person. I mean, if there existed one million computations for mankind alone, you should just say “oops” and admit that moral values are as arbitrary and personal as preferences about food. Wait, even more arbitrary! I mean, almost everyone likes ice-cream.

Non-cognitivistic theories like Emotivism,  Quasi-Realism or Prescriptivism are further examples of moral anti-realism.

In short: Almost all alien species and many humans have different preferences and can’t agree on what is “right”.

3. Moral Reductionism vs. Non-Reductionism

Let’s assume that moral facts exist. Are they reducible to non-moral facts? The official terms are very vague. Some people e.g. think that Railton’s metaethics is non-reductionistic whereas e.g. Miller and I believe that Railton is obviously a moral reductionist.

P (Moral Reductionism) + P(Moral Non-Reductionism) = 1 – P (Moral Anti-Realism)

3.1 Moral Reductionism (40%)

Description: Moral facts are reducible to non-moral facts. These moral facts are therefore not supernatural or facts sui generis because moral reductionism presupposes moral naturalism. I guess Yudkowsky’s metaethics is an example of moral reductionism because moral facts are reducible to a specific computation which properties can be extracted from the human brain. Railton’s metaethics or Jackson’s analytical moral functionalism are also good examples.

3.2 Moral Non-Reductionism (25%)

Description: Moral facts are not reducible to non-moral facts. This doesn’t mean that those facts are somehow supernatural or sui generis. There are e.g. the Cornell Realists who think that moral facts are natural but also irreducible whatever that means.

With my Rationality-goggles on, Non-Reductionism seems totally wrong, but in section 4.2 I give you my main argument for “mysterious” theories.

4. Naturalism vs. Non-Naturalism

P(Naturalism) + P(Non-Naturalism) = 1 – P(Moral Anti-Realism)

(You also could define Moral Naturalism so as to include Moral Anti-Realism.)

4.1 Naturalism (40%)

Description: Moral facts are natural properties. There is nothing spooky or queer about them, they are part of the natural world as investigated by the natural sciences (including psychology).

4.2. Non-Naturalism (25%)

Description: Moral facts are not natural, but e.g. supernatural or somehow outside the “realm of normality” whatever that means. Most Non-Naturalists endorse some kind of “intuitionism”. Because moral properties are non-natural and we can’t detect them by our usual, natural senses, we must have some kind of special mental faculty that is capable of detecting those facts.

Again, with my Bayesian-goggles on this sounds quite ridiculous, but here is the main argument for Non-Naturalism or Non-Reductionism, as I understand it, which goes like this:

“Look. We know our explanations and theories don’t make much sense, but this isn’t our fault. The universe is fundamentally weird. Deal with it. And don’t give me that It-All-Adds-Up-To-Normality-Bullshit! Just take some LSD, DMT, meditate or, better yet, think about the implications of physicalism, Tegmark’s Ultimate Ensemble/MWI, acausal trade, TDT/UDT, FAI and stuff,  and you’ll see that your worldview is even more insane than ours!”

Admittedly, this whole argument reeks of Dark Side Epistemology, but sometimes I think it’s rather convincing.


Metaethics is rather useless. I’ll have to outline my thoughts on normative ethical theories in order to get some practical results that could eventually serve as a guide for the all-important question “What the fuck should I do with my life?“.

This entry was posted in Fundamentals, meta-ethics. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s