Infinite Ethics and Runaway Inquiry
The problem
The argument "even if objective value is very unlikely, missing it could have enormous cost, so inquiry is warranted" becomes dangerous when written as simple expected value. If a probability of 10^-100 multiplied by a huge payoff can dominate every decision, nearly any bizarre value hypothesis can be rationalized in the same way.
Even hypotheses such as "true value requires maximizing suffering" become impossible to ignore if assigned sufficiently large stakes. Once positive and negative infinities enter, ordinary expected utility may not even be well-defined.
The asymmetry of normative stakes is not simple expected value
The claim that nihilism has no objective stakes while realism may contain unknown stakes should be distinguished from "tiny probability times huge utility beats everything." The Core does not infer an all-in bet on one enormous value hypothesis. It infers only the weaker policy of not irreversibly closing routes of inquiry.
A 10^-100 chance of infinite good therefore does not by itself justify spending all current resources. Inquiry resources allocated to poorly evidenced hypotheses can themselves remain small and proportional to evidence.
Implications for the Core
The Core should not be formulated as straightforward maximization of "small probability × huge value." The emphasis should instead lie on local robustness: reversibility, information value, and preservation of routes for further inquiry. Resources devoted to inquiry also need limits so that extreme hypotheses cannot capture the entire decision process.
Open problems
- How can finite decision rules handle hypotheses containing infinite value?
- Is any threshold for discarding extremely low-probability hypotheses itself arbitrary?
- Does option value ultimately still require a value function?