1. Using climate model robustness as an example of scientific confirmation
Matt Newman
CIRES

2. Can we prove a scientific hypothesis?
All scientific knowledge is tentative and provisional, and nothing is final.  There is no such thing as final proven knowledge in science. 
The currently accepted theory of a phenomenon is simply the best explanation for it among all available alternatives.
As opposed to mathematical proofs, which are
final: once a theorem is proven it will forever be true
“binary”: a theorem is either true or not true
So we require all hypotheses to be testable
But we don’t “prove”, although we can disprove

3. How can we test and confirm a climate model?
A model represents a part of the world for some purpose
For example, a climate model represents the tropical Pacific to help us predict its evolution over the next year
A model is an expression of a theoretical hypothesis that we would like to confirm
So we similarly require a model to be testable.
Make a prediction and see if observations match the prediction
What if we can’t test our prediction in practice?
For example, repeat last 100 years of Earth’s history without increase in CO2
We could still require models (as a group) to be testable
Make a prediction with many models and see if they agree
Note that there could be many “equally good” models

4. Robustness in climate modeling
Climate scientists see the convergence of climate models on a result as confirmatory.
This is of practical importance, since getting all the details right in the model is impossible

5. Robustness (cont.)
Climate scientists see the convergence of climate models on a result as confirmatory.
This is of practical importance, since getting all the details right in the model is impossible
Philosophers of science tend to think that such convergence, or “robustness,” is not confirmatory, because the models could converge and still all be wrong.
“Confirmation bias” – answer is known ahead of time so only results that match are accepted
Unless: the result is the same in all (possible) models (eg, complete set of models)
Lloyd is claiming that robustness, in the sense in which climate science uses it, can by itself be used to confirm models (not just in climate science)

6. Model robustness
All other things being equal, if the “common causal core” exists, then some robust property will result
For example:
if increased greenhouse gases interact under physical laws (eg, conservation of mass, energy) on the Earth
then increased global mean temperature will result
But: how can we be sure that greenhouse gases were the relevant cause?
Sufficiently heterogeneous set of models all produce similar result
Heterogeneous as in different modeling centers, parameterizations, simple/complex, etc.
(Note that having models give same result isn’t enough: we need same cause too)

7. What does Lloyd mean by model robustness
Models yield similar predictions and “retrodictions”
Note that this does not require actually testing T against observations
But: robustness can also be in the evidence for the model’s common core
For climate models, empirical evidence for model parameterizations not its predictions
“Model robustness involves both the causal core of the individual models and the convergence of the model outcomes”
Contrast with
“measurement robustness” (eg, Avogadro’s number, requires independent tests)
“derivational robustness” (eg, random parametric insensitivity)

8. What does Lloyd mean by model robustness
Various models, which all include a given causal core (say, greenhouse gas increase) but otherwise have different components, all yield the same conclusion T
There are separate lines of evidence linking the causal core with T
There is independent empirical evidence for T
There is evidence for each of the different model components but not for which is “best”
Therefore, T is a robust result

9. Model causal cores would be robust if tuning doesn’t impact outcome and if tuning values still have empirical support (latter is key for Lloyd’s argument).
So what if tuning does impact outcome? Or what if it takes unsupported parameter values to get “good” outcome?

10. Or what if the models are not independent?

11. Or what if the models are not independent?
Or what if they agree but are all wrong? There are well- known model errors that are common to all models, such as equatorial “cold tongue” bias
observations
longitude
Model simulations

12. Or what if the models are not independent?
Or what if they agree but are all wrong? [There are well- known model errors that are common to all models.]
An alternative view: models are "a work of fiction…(where) some properties ascribed to objects in the model will be genuine properties of the objects modeled, but others will be merely properties of convenience.” ” (Cartwright)

13. Q: Is robustness sufficient to confirm models. Consider:
Q: Is robustness sufficient to confirm models? Consider: How much of a causal core is needed? Can models get a variety of things wrong and still be confirmatory if they get a variety of things right?
Class exercise: divide into two groups. One side takes affirmative, the other negative. Discuss within each group your position for about 10 minutes, then open up to group discussion.