191 Drawing Statistical Inference from Simulation Runs......the "fun" stuff!

Slides:



Advertisements
Similar presentations
ETM 607 – Output Analysis: Estimation of Absolute Performance
Advertisements

Experimental Design, Response Surface Analysis, and Optimization
Sampling Distributions (§ )
Output analyses for single system
Spreadsheet Demonstration New Car Simulation. 2 New car simulation Basic problem  To simulate the profitability of a new model car over a several-year.
1 Statistical Inference H Plan: –Discuss statistical methods in simulations –Define concepts and terminology –Traditional approaches: u Hypothesis testing.
Output Data Analysis. How to analyze simulation data? simulation –computer based statistical sampling experiment –estimates are just particular realizations.
Simulation Where real stuff starts. ToC 1.What, transience, stationarity 2.How, discrete event, recurrence 3.Accuracy of output 4.Monte Carlo 5.Random.
Analysis of Simulation Experiments
Output Analysis and Experimentation for Systems Simulation.
The AutoSimOA Project Katy Hoad, Stewart Robinson, Ruth Davies Warwick Business School WSC 07 A 3 year, EPSRC funded project in collaboration with SIMUL8.
Automating the Analysis of Simulation Output Data Katy Hoad, Stewart Robinson, Ruth Davies SSIG Meeting, 24th October 2007
Lecture 9 Output Analysis for a Single Model. 2  Output analysis is the examination of data generated by a simulation.  Its purpose is to predict the.
The AutoSimOA Project Katy Hoad, Stewart Robinson, Ruth Davies Warwick Business School OR49 Sept 07 A 3 year, EPSRC funded project in collaboration with.
EEM332 Design of Experiments En. Mohd Nazri Mahmud
Inferences About Process Quality
1 Simulation Modeling and Analysis Output Analysis.
BA 275 Quantitative Business Methods
Sample size. Ch 132 Sample Size Formula Standard sample size formula for estimating a percentage:
BA 427 – Assurance and Attestation Services
1 Terminating Statistical Analysis By Dr. Jason Merrick.
Simulation Output Analysis
Steps in Using the and R Chart
Estimation Basic Concepts & Estimation of Proportions
Steady-State Statistical Analysis By Dr. Jason Merrick.
Verification & Validation
Chapter 11 Output Analysis for a Single Model Banks, Carson, Nelson & Nicol Discrete-Event System Simulation.
Lecture 12 Statistical Inference (Estimation) Point and Interval estimation By Aziza Munir.
CS433 Modeling and Simulation Lecture 16 Output Analysis Large-Sample Estimation Theory Dr. Anis Koubâa 30 May 2009 Al-Imam Mohammad Ibn Saud University.
Statistical Decision Making. Almost all problems in statistics can be formulated as a problem of making a decision. That is given some data observed from.
University of Ottawa - Bio 4118 – Applied Biostatistics © Antoine Morin and Scott Findlay 08/10/ :23 PM 1 Some basic statistical concepts, statistics.
IE 429, Parisay, January 2010 What you need to know from Probability and Statistics: Experiment outcome: constant, random variable Random variable: discrete,
2.6 Confidence Intervals and Margins of Error. What you often see in reports about studies… These results are accurate to within +/- 3.7%, 19 times out.
10.1: Confidence Intervals – The Basics. Review Question!!! If the mean and the standard deviation of a continuous random variable that is normally distributed.
Copyright ©2011 Nelson Education Limited Large-Sample Estimation CHAPTER 8.
Brian Macpherson Ph.D, Professor of Statistics, University of Manitoba Tom Bingham Statistician, The Boeing Company.
Decision making Under Risk & Uncertainty. PAWAN MADUSHANKA MADUSHAN WIJEMANNA.
1 Terminating Statistical Analysis By Dr. Jason Merrick.
11/18/2015 IENG 486 Statistical Quality & Process Control 1 IENG Lecture 07 Comparison of Location (Means)
Chapter 8 : Estimation.
CHEMISTRY ANALYTICAL CHEMISTRY Fall Lecture 6.
Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.
CONFIDENCE STATEMENT MARGIN OF ERROR CONFIDENCE INTERVAL 1.
Confidence Interval Estimation For statistical inference in decision making:
1 OUTPUT ANALYSIS FOR SIMULATIONS. 2 Introduction Analysis of One System Terminating vs. Steady-State Simulations Analysis of Terminating Simulations.
1 BA 275 Quantitative Business Methods Quiz #2 Sampling Distribution of a Statistic Statistical Inference: Confidence Interval Estimation Introduction.
Ex St 801 Statistical Methods Inference about a Single Population Mean.
: An alternative representation of level of significance. - normal distribution applies. - α level of significance (e.g. 5% in two tails) determines the.
Ch 8 Estimating with Confidence 8.1: Confidence Intervals.
1 Probability and Statistics Confidence Intervals.
Sample Size Mahmoud Alhussami, DSc., PhD. Sample Size Determination Is the act of choosing the number of observations or replicates to include in a statistical.
Uncertainty and confidence Although the sample mean,, is a unique number for any particular sample, if you pick a different sample you will probably get.
CHAPTER 15: Tests of Significance The Basics ESSENTIAL STATISTICS Second Edition David S. Moore, William I. Notz, and Michael A. Fligner Lecture Presentation.
Descriptive Statistics Used in Biology. It is rarely practical for scientists to measure every event or individual in a population. Instead, they typically.
Lesson 18: Sampling Variability and the Effect of Sample Size Students use data from a random sample to estimate a population mean. Students know that.
Statistical Decision Making. Almost all problems in statistics can be formulated as a problem of making a decision. That is given some data observed from.
K. Salahpp.1 Chapter 9 Output Analysis for Single Systems.
ECO 173 Chapter 10: Introduction to Estimation Lecture 5a
R. E. Wyllys Copyright 2003 by R. E. Wyllys Last revised 2003 Jan 15
Al-Imam Mohammad Ibn Saud University Large-Sample Estimation Theory
CPSC 531: System Modeling and Simulation
ECO 173 Chapter 10: Introduction to Estimation Lecture 5a
Statistical Methods Carey Williamson Department of Computer Science
CONCEPTS OF ESTIMATION
(or why should we learn this stuff?)
SAMPLE DESIGN: HOW MANY WILL BE IN THE SAMPLE—DESCRIPTIVE STUDIES ?
Carey Williamson Department of Computer Science University of Calgary
MECH 3550 : Simulation & Visualization
Range, Width, min-max Values and Graphs
Sampling Distributions (§ )
Presentation transcript:

191 Drawing Statistical Inference from Simulation Runs......the "fun" stuff!

192 Essential Issues u Warm-up Period u Replication Determination

193 THE NONTERMINATING SIMULATION THE NONTERMINATING SIMULATION

194 Warm-up Period u Initial transient period. u When steady state is achieved u Determines deletion amount u Graphical procedure

195 Definitions W = Warm-up (transient) period L = Length of replication n = Number of replications m = Moving average period

196 Procedure u Select decision variable. (e.g. Quantity of WIP) u Make n>4 replications of duration L, where L is much larger than expected value of the transient period. u Prepare a table of observations.

197 Procedure u Calculate the average across each of the n periods (e.g. all day 1s, all day 2s, etc.). u Plot a moving average for m = 5, m = 10 or m = 15. u Select the warm-up period W that shows the smallest deviation from a straight line average.

198 Daily Number in WIP Rep 1 Rep 2 Rep 3 Rep n Daily Avg * Day 1 Day 2 Day 3 Day 30 * Averages are hypothetical assuming all observations are considered. etc

Daily Avg Moving Avg

Periods (days) Moving Average m = 5

Periods (days) Moving Average m = Convergence

202 Clues to Success u Use n > 4 replications initially. u Keep m > L/2. u Increase replications n rather than length of run L to achieve greater smoothness. u Better to choose a warm-up period too long than too short.

203 THE TERMINATING SIMULATION

204 How Many Replications 10 ? 50 ? 100 ?

205 A “Quick” Method u Select your decision variable. (e.g. Average process time) u Decide how close you would like to be. “I’d like to be within +/-3 minutes of the actual process time…” “I’d like to be within +/-3 minutes of the actual process time…”

206 A “Quick” Method u “Estimate” the standard deviation of your decision variable... “I believe the maximum and minimum process times are 65 and 15 minutes respectively.” “I believe the maximum and minimum process times are 65 and 15 minutes respectively.” = 50 S = 50/6 = 8.33

207 A “Quick” Method u Select precision level. “I want to be 95% certain of my answer” (i.e. a =.05). “I want to be 95% certain of my answer” (i.e. a =.05). For 95% confidence interval, Z = 1.96.

208 A “Quick” Method u Solve... n = ( ) Z * S 3 n = ( 1.96 * 8.33 ) 2 2 n = 31 3

209 Precision Level Differences How Close 90% 95% 99% to Actual Mean (Min)

210 Or Use Stat::Fit

211 Open Stat::Fit from the Tools menu Open Stat::Fit from the Tools menu

212 Use the Utilities - Replications menu and fill-in the information as we did in the previous example Use the Utilities - Replications menu and fill-in the information as we did in the previous example

213 Number of replications Number of replications

214

215

216