1. Frédéric Haldi, Darren Robinson, Claus Pröglhöf, Ardeshir Mahdavi
A partial double blind evaluation of a comprehensive window opening model
Frédéric Haldi, Darren Robinson,
Claus Pröglhöf, Ardeshir Mahdavi
Ecole Polytechnique Fédérale de Lausanne (EPFL)
Vienna University of Technology
Vienna, BauSIM 2010 Conference

2. Motivation
Predicting occupants’ behaviour is crucial for accurate energy / comfort predictions from building simulation programs.
Models for the prediction of actions on windows, shading devices and electrical lighting have been developed.
Lack of validation. In general no estimation of predictive accuracy has been provided.
Validation on external data. The most conclusive evaluation can be achieved if two different data samples are used for its derivation and validation.
Independent verification. In order to avoid any bias, model development and validation should be performed by separate groups.

3. Models
Three approaches are used to model occupants’ actions on windows:
Bernoulli process: At each time step, the probability of observing a window to be open is independently determined by a probability (logistic model with qout and qin).
Markov chain: At each time step, window openings and closings are modeled by logistic transition probabilities.
Hybrid model: The above Markov chain is extended to a continuous-time process based on a Weibull distribution for opening durations.
F. Haldi, D. Robinson, Interactions with window openings by office occupants, Building & Environment, 44(12), , 2009.

4. Field surveys Survey in Lausanne, Switzerland
Eight years of measurements in the Solar Energy and Building Physics Laboratory experimental building.
Observations in 14 cellular offices of the south façade.
Each occupant may act on a window.
Survey in Hartberg, Austria
Six months of in situ measurements.
Observations in 6 offices of the north façade.

5. Verification procedure
The experimenters agreed on a preliminary schedule and validation criteria.
Internal validation A. Having measured the dataset A, the experimenter develops predictive models from these data and uses half of the dataset as a training set for model calibration and the remaining part as a validation set.
External validation A to B. The experimenter B provides the experimenter A with the set of potential driving variables of the dataset B. Based on the model calibrated with the dataset A, the experimenter A runs simulations of the window states of dataset B, which are sent to experimenter B.
Internal validation B. The experimenter B sends a part of his dataset (including window states) to experimenter A, who performs simulations of the unknown window states in dataset B.
External validation B to A. Finally, having received the whole dataset B, the experimenter A calibrates a model and simulates windows states of dataset A.

6. Validation criteria Global indicator:
Overall ratio open: Fraction of instances with open windows.
Discrimination indicators:
Truly positive: Fraction where a window is correctly predicted to be open.
Truly negative: Fraction where a window is correctly predicted to be closed.
Falsely positive: Fraction where a window is wrongly predicted to be open.
Falsely negative: Fraction where a window is wrongly predicted to be closed.
True prediction: Fraction where the state of a window is correctly predicted.
Action indicators:
Proportion of correctly predicted opening/closing actions: Fraction of actions with a temporal match between observed and simulated opening/closing actions (within ±30’).

7. Prediction of Swiss from Swiss Prediction of Swiss from Austria
Predicting Swiss data
Prediction of Swiss from Swiss
In both cases, superiority of the hybrid model regarding the proportion of correct predictions.
Low ability to reproduce the exact timing of opening and closing actions.
Small decrease of predictive accuracy when using external data.
Prediction of Swiss from Austria
Predicting Swiss data

8. Predicting Austrian data
Prediction of Austria from Austria
Predicting Swiss data
Much smaller general ability to predict the state of windows
No strong decrease in the quality of predictions when using external data for calibration.
Possible causes: smaller dataset, less actions on windows
Prediction of Austria from Swiss
Predicting Austrian data

9. Complementary analyses
LESO model also reproduces well behaviour for a Swiss residential building.

10. Conclusion
First step toward rigorous evaluation of predictions from stochastic window operation models: general model formulation, based on local stimuli.
Good ability to reproduce the state of windows, relatively poor ability to reproduce the timing of actions (as expected).
Similar performance between validation runs on internal and external data: model appears to be robust!
Further research to be done:
Predictive accuracy for other climates.
Accuracy of models for other building controls (lights, shading devices) and a wider sample of buildings.
We need more data for further model calibration / verification!