Sometimes it’s such a strange thought that the people in the history books really lived and experienced and died – that there’s so much more depth to history than anything I’ve seen with my own eyes; so much more life than anything I’ve lived.
Suppose that Tegmark’s Mathematical Universe Hypothesis is real.
What fraction of sentient species that try to follow the policy “Ban all physics experiments involving situations with a remote possibility of being novel, until you can augment your own intelligence enough to do error-free cognition”;
And what fraction of sentient species that go ahead and do physics experiments;
Survive in the long term, on average?
It’s of course politically infeasible to implement a “stop-all-novel-experiments”-policy.
There’s a social norm in favor of “diversity”, but not diversity. There’s a social norm in favor of “free speech”, but not pornography. There’s a social norm in favor of “democracy”, but it doesn’t spontaneously occur to most people to suggest voting on their arguments. There’s a social norm in favor of “love”, but not for letting some damn idiot marry your daughter even if the two of them are stupid and besotted.
Our society values “truth” (truthiness) more than truth, i.e. correspondence between beliefs and reality.
…I generally advise against making up probabilities, unless it seems like you have some decent basis for them. This only fools you into believing that you are more Bayesian than you actually are.
I would advise, in most cases, against using non-numerical procedures to create what appear to be numerical probabilities. Numbers should come from numbers.
Great comment by Robin Hanson:
“I very much disagree with this quote, and much of the rest of the post. Most of our reasoning about social stuff does not start from concrete numbers, so this rule would forbid my giving numbers to most of what I reason about. I say go ahead and pick a number out of the air, but then be very willing to revise it upon the slightest evidence that it doesn’t fit will with your other numbers. It is anchoring that is the biggest problem. Being forced to pick numbers can be a great and powerful discipline to help you find and eliminate errors in your reasoning.”
I agree with Hanson. In fact, I think about posting my probability-estimates on a wide range of topics (like meta-ethics, theoretical physics, philosophy, etc.) in the following months.
Furthermore, you have to assign probabilities to moral theories in order to be able to use the Parliamentary-Model by Bostrom and Ord. And since I’m getting more and more confused about morality, rationality, ontology, physics, well, essentially everything, I have to use something like the Parliamentary-Model. (Moral) uncertainty is a pain in the ass but we have to deal with it somehow.