Ethics: Weak Ordering

Objectivity

Whether ethics is "objective" is a contentious issue - largely, I feel, because people don't lay out what they mean by "objective". Here's my preferred definition:

Given that a superintelligent AI could be programmed to only care about paperclips Paperclip maximizer, their naively doesn't seem to be any link between intelligence and any particular value system Bostrom, N.

However, I believe there are a large number of ethical systems that would be ruled out by sufficient knowledge and intelligence. In particular, ethical systems that are logically contradictory cannot be correct, and so, we can assume that any sufficiently intelligent and experienced being wouldn't believe them. Likewise, there are properties like temporal consistency Time consistency that any reasonable person would accept as good in any useful decision-making procedure, ethical or not.

Moreover, humans tend to have at least somewhat similar intuitions regarding what constitutes ethical and unethical behavior. My question is this: after crossing out all logically contradictory ethical systems, what systems remain that adhere at least somewhat to human intuition? If there is only one, then I feel comfortable calling it "objective"; otherwise, I fear we will be able to make little uncontroversial progress.

Unsurprisingly, I claim to have some knowledge of what this system is - what can I say, humility was never my strong point. However, I don't think I can convince you that ethics, as a concept, is meaningful if you don't already believe so. I take that as a starting assumption.

Starting Assumptions

I’m going to assume that an ethical system is a way to determine which of a set of actions to perform within a particular situation, including the ability to say that multiple actions are "equally good". To be more precise, I’m going to define ethics as a complete and transitive ordering of actions, meaning that it satisfies

1. completeness - either action $X$ is better than $Y$, $Y$ is better than $X$, or both actions are equivalently good.
2. transitivity - if $X$ is better than $Y$, and $Y$ is better than $Z$, then $X$ is better than $Z$.

These assumptions allow us to rank actions, but also allow for ties. This, allows us to adopt some slightly mathy notation:

• $X \gt Y$ denotes "$X$ is better than $Y$"
• $X \geq Y$ denotes "$X$ is better than or as equally good as $Y$"
• $X \backsim Y$ denotes "$X$ and $Y$ are equally good"

One thing that mathematically follows from this is that given a situation, we can create a "weak ordering" on all the actions in that situation. Economists call this a "rational preference relation" Preference (economics). To my mind, these "requirements" aren't really assumption so much as they are implicit in any conception/discussion of ethics.

Given a situation, each action will lead to a sequence of future instances - that is, a "universe branch". Even if you think there is true randomness in the universe, it is still the case that, in the future, one particular branch will occur. Conversely, every possible universe branch corresponds to exactly one action in that situation, because that action is encapsulated in the universe branch. Hence, there is a one-to-one correspondence between actions-situation pairs and universe branches. This means an ethical system must be equivalent to defining a weak ordering on all possible universe branches.

Generally, when I bring up this "weakly ordered" conclusion people either

1. don’t know what a "weak ordering" is
2. reject completeness
3. reject transitivity
4. think this is so obvious as to not be worth mentioning

Let's go through all three responses.

Weak Ordering

When I say that a coherent ethical system requires a weak ordering on universe branches, what I'm really saying is that we can rank all possible universe branches. Because there are, presumably, infinitely many universe branches, it doesn't neccessarily follow that there is a "best" or "worst" branch, just as there isn't a smallest and largest number. However, it does mean that I can (theoretically) rank all these branches by "goodness" while allowing for ties.

Completeness

Some people don’t like the conclusion that we can arrange universes on a line like this, and work backwards to reject completeness. This means they think that there are two univeres-branches that are incomparable.

This may seem reasonable, but this implies it is impossible to choose which action or set of actions to perform. This is equivalent to a rejection of ethics itself, which, as I've already stated isn't a line of reasoning we're going to be pursuing.

Transitivity

Other people don’t like the conclusion that we can arrange universes on a line like this, and work backwards to reject transitivity. However, there is a very good reason to avoid this.

Suppose Alice always does the right thing. Suppose Bob is choosing among a set of choices $A$, $B$, and $C$. If we reject transitivity, then there are some branches $A$, $B$, and $C$ represented ethically by $A \gt B \gt C \gt A$.

If it seems really weird, that we have this cycle, that's good. I agree. These cycles exist if (and only if) we reject transitivity. If it doesn't seem weird, let me explain why it should, and why transitivity is an essential property for any reasonable ethical system to have.

Suppose, that Bob initially wants to choose $A$. He tells Alice "if you give me \$1, I'll switch to $B$." Since Alice is perfectly ethical, she agrees. Bob will repeat this offer to get another dollar to switch from $B$ to $C$. Finally, Bob will make this offer one more time to switch from $C$ to $A$. In this way, Bob will have made \$3 from Alice for doing nothing. He can continue this indefinitely, until he has all of Alice's money and does nothing in return.

The problem with ethical systems that aren't transitive is that they are exploitable in this way. Likewise, if you think ideal people shouldn't be this trivially exploitable, you are implicitly accepting transitivity.

Not Obvious

Finally, some people think completeness and transitivity are so obvious as to not be worth mentioning. However, formally accepting completeness and transitivity already allows us to prove some fairly interesting things. Let me give an example.

Imagine you are God and are trying to choose which of four possible universes to create. All the universes contain people, each stranded alone on an island. Here are the universes you're considering:

1. Alice is very happy; Bob is moderately happy.
2. Bob is moderately happy.
3. Bob is very happy.
4. Bob is very happy; Carol is moderately happy.

Pretty much everyone should agree that (2) is worse than (3), because it is good for people to be happier.

I imagine many people would further like to claim that universes (1) and (2) are equally good, because it seems like the decision of whether to give birth to someone is ethically neutral. I'm not saying that *you* think this, but you shouldn't have trouble imagining people who do.

By an identical argument, those same people would think that (3) and (4) are equally good.

Finally, it is very clear that options (1) and (4) are equally good, because they differ only in people's names. This leads us to

This violates transitivity! We're saying that 1, 2, 3, and 4 are all equivalent, but also that 2 is worse than 3!

In this way, we've proven that you can't simultaneously believe

1. Deciding whether to give birth to a someone is always morally neutral.
2. Making people happier is good.

This example should demonstrate that even these two obvious and minimal ethical assumptions are already yielding some insight - telling us that some beliefs at mathematically inconsistent. Let's see what happens if we add a couple more...