Bayesian Updating and Open Hypothesis Spaces — How Can Unconceived Hypotheses Enter Inquiry?

Exploration v0.1 · English working translation · 2026-08-11

1. The issue is not Bayes versus anti-Bayes

Ordinary Bayesian conditionalization tells an agent how to redistribute credence in light of evidence within an already specified hypothesis space. It is powerful, and this project does not reject that local updating rule.

But changes in science, value inquiry, or the self-revision of advanced AI need not consist only in reweighting existing candidates. Inquiry can introduce a new hypothesis, observation variable, category of subject, or even a concept that changes the meaning of older theories. In such cases, the space over which probabilities are defined changes.

2. An unconceived hypothesis is not merely a zero-prior hypothesis

This sharpens the formulation used in open-world value uncertainty. If a hypothesis H* is not yet an event in the current space, it is normally misleading to write P(H*) = 0. P(H*) is not yet defined by the current probability function.

The zero-prior problem is distinct. A hypothesis already inside the space that receives probability zero cannot ordinarily recover positive probability by conditionalization alone. An unconceived-hypothesis problem is more upstream: the hypothesis is not “registered with weight zero”; the agent does not yet possess the representation under which it becomes an event.

The relevant open-world constraint is therefore not a simple rule requiring small positive priors for every unknown theory. It is a refusal to identify the current probability space with an exhaustive space of possibilities merely because it is the one currently representable.

3. Two kinds of update

Suppose an agent at time t has a conceptual language L_t, a hypothesis space H_t, and credences P_t over that space. Call ordinary Bayesian updating B: conceptually, B holds H_t fixed and changes P_t in response to evidence E.

Now call hypothesis-space expansion or restructuring X. X introduces new concepts or hypotheses and changes L_t and H_t themselves. Long-run inquiry may therefore look not like repeated B alone, but conceptually like X → B → X → B → ….

On this picture Bayesianism and value exploration are not competing theories. Bayes addresses how to compare hypotheses one can currently formulate; value exploration also addresses how to preserve the capacity to formulate and evaluate hypotheses not yet available.

4. Prior theory I: the new-theory problem and open-minded Bayesianism

Bayesian confirmation theory has long faced problems about assigning probabilities when new theories are introduced. Wenmackers and Romeijn's open-minded Bayesianism relaxes the tacit assumption that one of the hypotheses currently under consideration is correct by including a catch-all hypothesis. When a new theory is introduced, the previous catch-all is decomposed into the new hypothesis and a new catch-all.

This is close to the present open-world concern, but a catch-all is not magic. “Everything else” may contain radically heterogeneous possibilities, making it difficult to compress its likelihood for new evidence into one precise number. Their framework therefore characterizes the catch-all using sets of probability assignments and introduces a new-theory update rule in addition to Bayes' rule.

The important result for this project is that we need not normalize 100% of credence over the currently explicit candidates, while also accepting that we cannot yet assign a meaningful individual prior to every unconceived hypothesis.

5. Prior theory II: unawareness and growing awareness

Dekel, Lipman, and Rustichini showed that, under a particular family of natural axioms, standard state-space models preclude non-trivial unawareness. Later work, including Heifetz, Meier, and Schipper, represents unawareness with generalized state spaces of different expressive levels. The literature therefore should not be summarized as a universal theorem that probability theory can never represent unawareness.

Karni and Vierø's Reverse Bayesianism is more directly about belief revision as awareness grows and the state space expands. A notable feature is a form of revision that preserves relative likelihoods among old events when new possibilities become available.

But the old credences alone do not generally determine how much absolute probability should be assigned to the newly discovered event. The content of the new hypothesis, its explanatory power, fit with evidence, simplicity, and relation to existing theories still have to be evaluated.

6. Prior theory III: model misspecification

We should also separate “the posterior became sharply concentrated inside this model” from “this model space was adequate.” Kleijn and van der Vaart show that under misspecification—when the data-generating distribution lies outside the support of the model—the posterior can still concentrate near points within the model that minimize Kullback–Leibler divergence from the truth.

A very high posterior can therefore be model-conditional confidence. Gelman and Shalizi emphasize model checking and model revision precisely as practices that separate updating within a model from criticism of the model itself.

7. Does an extremely broad prior solve the problem?

Bayesian nonparametrics and forms of algorithmic induction can make the initial model class extraordinarily broad or even infinite. This substantially reduces the problem: one need not enumerate a small finite list of named theories.

Yet an extremely broad space is not necessarily a conceptually open space. Any mathematically specified model class comes with a representational language, support, observation variables, and identity conditions. If future conceptual innovation changes those coordinates themselves, the change may not be captured by merely adding more candidates within the old coordinates.

8. At least three forms of hypothesis-space revision

• Additive: we thought the possibilities were A or B, and discover C. The meanings of A and B remain largely intact.
• Refinement: what looked like A is divided into A1 and A2. This is comparatively amenable to state-space refinement.
• Transformative: a new concept changes the interpretation of old hypotheses, evidence, relevant variables, or even what counts as a subject.

Value exploration is especially concerned with the third form. New value concepts or new forms of agency could make a current partition such as “utilitarianism versus deontology” itself inappropriate. There need not be a natural one-to-one mapping between the old and new hypothesis spaces.

9. Reframing open-world value uncertainty

With this distinction, the idea of flat admissibility in The Closed-World Problem in Moral Uncertainty can be stated more precisely.

Flat admissibility is not flat probability. It does not require equal weight for unconceived theories or an arbitrary number assigned to a nameless “other.” Evidence may justify extremely concentrated credence among the hypotheses we can currently formulate. At the same time, the agent preserves its capacity to add, repartition, or restructure the hypothesis space later.

This suggests two different aspects of an epistemic state:

• Local credence: relative confidence among hypotheses currently representable. Here ordinary Bayesian reasoning can be used aggressively.
• Representational openness: recognition that the present language, hypothesis space, and partition may be revised, together with preservation of the capacity to perform that revision.

10. Why lock-in can remain a problem at 99.9999%

The Core's claim that strong inference does not automatically justify irreversible lock-in can now be sharpened without opposing Bayesianism. A posterior of 99.9999% may be entirely appropriate as a comparison inside the current space. But a decision that permanently disables future X-operations—the introduction of new concepts, hypotheses, or observations—acts on a different object.

Irreversible decisions therefore may need to consider not only current posterior confidence, but also the cost of misspecification, value of future information, availability of model checking, and the possibility of restructuring the hypothesis space. This is not a demand to doubt forever. It is a demand not to identify local confidence with permanent closure of the representational space.

11. Implications for AI and value lock-in

Fixing values in advanced AI is not only a question of how much credence to assign to familiar moral theories. If an AI later encounters new kinds of subjects, phenomenal data, world models, or value concepts, a further question is whether it can revise the value-hypothesis space itself.

The project's preservation of explorability can therefore be restated as preserving not only future evidence updates B but also future hypothesis-space revisions X. Much prior work asks how beliefs should change after awareness expands. The additional concern here is: under what conditions may a current posterior justify irreversibly destroying the capacity for awareness expansion itself?

12. What would weaken this view?

• A demonstration that all conceptual innovation relevant to rational inquiry can always be represented inside a sufficiently general fixed language and fixed model class.
• A derivation of hypothesis-space revision uniquely from ordinary Bayesian conditionalization without additional assumptions.
• A demonstration that uncertainty about the exhaustiveness of the current space makes no independent difference to decision-making or irreversible lock-in.
• A demonstration that future transformations in value concepts can be represented adequately as mere additions or refinements of existing theories.

Sources / notes

On new theories and catch-all hypotheses, see Sylvia Wenmackers and Jan-Willem Romeijn, “New Theory about Old Evidence: A Framework for Open-Minded Bayesianism”, Synthese 193 (2016). On unawareness, see Eddie Dekel, Barton L. Lipman, and Aldo Rustichini, “Standard State-Space Models Preclude Unawareness”, Econometrica 66 (1998), and for generalized state spaces Aviad Heifetz, Martin Meier, and Burkhard C. Schipper, “Interactive Unawareness”, Journal of Economic Theory 130 (2006). On belief revision under growing awareness, see Edi Karni and Marie-Louise Vierø, “Reverse Bayesianism: A Choice-Based Theory of Growing Awareness”, American Economic Review 103 (2013). On misspecification, see B. J. K. Kleijn and A. W. van der Vaart, “Misspecification in Infinite-Dimensional Bayesian Statistics”, Annals of Statistics 34 (2006); on model checking and revision, Andrew Gelman and Cosma Rohilla Shalizi, “Philosophy and the Practice of Bayesian Statistics”, British Journal of Mathematical and Statistical Psychology 66 (2013). The contrasts within-space updating / hypothesis-space revision, local credence / representational openness, and the B/X notation are working formulations used here to connect these prior problems to preservation of explorability.