Xan left this gem in response (slightly re-worked):
We start out only dimly aware of the space of possible theories. When new data comes along, it makes us consider possible new theories that never even occurred to us before. And all along the way, our belief probabilities are calculated _as if_ we are currently pretty much aware of the entire space of theories.
There is a problem with treating science as equivalent to truth-seeking. Science is a fine tool for what it's trying to accomplish, but science is not rationalism.
If we want to believe things with the right level of confidence, we have to step outside of science and realize that the scientific method systematically promotes a very specific kind of output. Science does not peddle in counterfactuals for which there is no data. Science waits until it has a massive amount of data with a p-value of .000001, and then it makes strong statements that don't generalize outside of what it has data on. From this approach, we would _expect_ the kind of progression you have noticed. From the scientific perspective, as more datapoints become available, the space of theories-science-allows-itself-to-consider gets bigger.
