The Limits of Inferential Certainty About Value
1. The question is not whether value can be inferred
If value premises are included, further value claims can be inferred; and descriptive evidence about the world could in principle make moral realism the best explanation. This thesis does not claim that inference tells us nothing about value.
Nor is the relevant distinction simply between “inferential confidence” and “non-inferential foundation.” At least three epistemic positions should be distinguished: inferential convergence continuous with present cognition, inference whose epistemic structure itself has been transformed, and foundational cognition in which normativity is directly presented.
2. Inference continuous with present cognition cannot completely certify itself from outside itself
If the validity of an inferential rule R is proved by another inference, that proof itself uses some inferential rule. If it uses R or an equivalent rule, there is circularity; if it uses a different rule R2, the validity of R2 becomes the next question.
This is not an argument that inference should be distrusted. Coherence, predictive success, compression, cross-checking, and successful intervention can justify overwhelming rational confidence in this kind of inferential system. The point is that the confidence still arises within an epistemic web in which world-models, representations, and inferential rules mutually support and constrain one another.
The scope of the claim matters. This thesis does not prove that every possible intelligence and every possible form of inference must share the same limitation. If future agents acquire representational forms unavailable to us, genuinely independent routes of validation, or new relations among inferential systems, their epistemic status should be reassessed rather than assumed in advance.
Merely scaling capability does not by itself cross this boundary. An agent that performs structurally familiar reasoning at vastly greater speed, depth, and precision may reach overwhelming confidence, but if the relevant dependency structure remains the same, it is still continuous with present cognition. The important variable is not sheer capability but the epistemic structure of what validates what.
3. At least three epistemic positions for value cognition are conceivable
Foundational presentation: a hypothetical case in which value with normative force—beyond mere pleasure, pain, or preference—is directly given with epistemic strength comparable to the epistemic minimum.
Continuous inferential discovery: a case in which, after integrating evidence about consciousness, subjects, rationality, or cosmic structure, a theory positing objective value overwhelmingly outperforms its rivals as the best explanation. Even radically stronger intelligence belongs here if its dependence on world-models, representations, and inferential rules remains structurally continuous with present cognition.
Epistemically transformative inference: a case in which cognition is not merely faster or larger, but the structure of representation, validation, or grasp of possibility changes so deeply that the agent gains epistemic footholds unavailable to current subjects. Whether such cognition could attain a status comparable to foundational presentation, or would remain another kind of fallible inference, is left open.
Most forms of moral realism we can realistically investigate today belong to the second route. That does not make them weak. Like belief in the physical world, an inferential best explanation can in principle become so strong that it is treated as practically near-certain. The third route is kept open to avoid projecting the limits of current cognition onto all possible future cognition.
4. There are multiple bridges from phenomenal value to normativity
- There is an appearance of pain.
- The subject has negative valence or an avoidance tendency toward it.
- Other subjects have structurally similar appearances.
- There is reason to treat those subjects symmetrically across differences of identity.
- That symmetry is not merely a consistency convention but has universal normative force.
- Therefore rational agents have reason to reduce that pain.
These are not one proposition. In particular, the appearance of “badness” or aversion is not automatically identical to the appearance of a subject-independent ought.
5. The still-unknown structure of the world can matter to value inference
We need not say that no ethical judgment is possible until the entire universe is understood. But many ontological, consciousness-related, subject-related, and cosmological facts that could affect the truth conditions of value remain deeply unsettled.
The structure of consciousness, personal identity, treatment of copied subjects, artificial consciousness, an infinite universe, or the relation between experience and normativity could all change the meaning or consequences of the same ethical theory. A future theory of objective value should therefore, insofar as it remains within an inferential layer continuous with present cognition, be understood as a best explanation conditional on the world-model then available. If the form of cognition itself changes, we must reassess how much of that conditionality remains.
6. A best explanation can bind practice very strongly
Fallibility does not entail weak action. If one value theory overwhelms its competitors in explanatory power, prediction, and coherence, institutions and resource allocation may rationally shift very strongly toward it.
The proposed asymmetry is to let the strength of practical commitment rise with the evidence, while imposing a distinct threshold for irreversible closure of inquiry. Confidence of 99.9999% may justify very strong action; it does not automatically justify physically destroying every route by which the residual uncertainty could ever be examined.
7. Why skepticism can remain even when inferential value is strong
Skepticism here does not mean refusing to adopt a conclusion. It means adopting the current best explanation while retaining its error conditions. For a value theory at the present-style inferential layer, one keeps track of what could falsify it, which changes in the world-model would alter the conclusion, and how it might appear under different structures of agency or cognition.
In this sense, appropriate skepticism is not the enemy of value realization. It is an error-correction mechanism in case the theory of value is wrong.
8. The asymmetry of normative stakes
If objective value does not exist, there is—objectively—no value difference between inquiry and non-inquiry. If objective value does exist, however, there may be unknown and extremely consequential normative differences. The nihilist hypothesis therefore does not itself provide an objective reason that inquiry ought to stop.
To connect this asymmetry to action, however, one still needs a conditional meta-attitude: if my objectives are justifiable, I want objective choice and revision to remain responsive to that justification. This thesis does not hide that additional commitment.
9. A correction to the simple slogan “uncertainty implies inquiry”
In an earlier, weaker formulation, corrigibility, information value, and avoidance of irreversible loss had to be introduced as independent meta-preferences. The revised view still needs them, but gives them a narrower role.
Preserving explorability is not a substitute for value itself. It is an error-correction strategy adopted by an agent that keeps the question of objective justification open and wants objective choice to remain responsive to it while the epistemic status of value remains inferential in the present-style sense. If foundational presentation, or epistemically transformative inference, were to satisfy sufficiently strong stopping conditions, this strategy could itself be withdrawn.
10. Implications for ASI
A superintelligence might understand the physical world, its own formation, human evolutionary history, and every known ethical theory without thereby receiving foundational normativity directly. At the same time, it might use vastly stronger inference than humans to discover objective value as the best explanation.
Here too, scale of inferential capability should be separated from change in epistemic form. Massive computational power can radically increase confidence through continuous inference without thereby escaping the present-style dependency structure. Conversely, an ASI that acquires new forms of representation, validation, or self-evaluation may approach the third route.
Three roles for ASI therefore remain open: first, drastically raising continuous inferential confidence in value theories; second, opening modes of cognition unavailable to humans; and third, testing whether those new modes are merely another fallible form of inference or instead have an epistemic status capable of closing the relevant space of live alternatives.
11. What would weaken this thesis?
- A demonstration that phenomenal value analytically or directly contains subject-independent normativity.
- A general principle showing that inferential systems continuous with present cognition can terminally self-validate despite retaining dependence on world-models, representational schemes, and inferential rules.
- A complete determination of every aspect of world structure relevant to the truth conditions of value, leaving only one value theory as necessary.
- A general principle showing that present-style inferential convergence alone can guarantee that no future heterogeneous form of cognition could recover a genuine competing possibility.
- A demonstration that distinguishing foundational presentation and epistemically transformative inference still leaves no meaningful difference between inferential best explanation and terminal epistemic closure in value cognition.