Chapter 12 Review of Calculus and Probability to accompany Operations Research: Applications and Algorithms 4th edition by Wayne L. Winston Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Description A review of some basic topics in calculus and probability, which will be useful in later chapters.

12.1 Review of Differential Calculus The limit is one of the most basic ideas in calculus. Definition: The equation means that as x gets closer to a (but not equal to a), the value of f(x) gets arbitrarily close to c. It is also possible that may not exist.

Example 1 Show that Show that does not exist. Solution To verify this result, evaluate x2-2x for values of x close to, but not equal to, 2. To verify this result, observe that as x gets near 0, becomes either a very large positive number or a very large negative number.. Thus, as x approaches 0, will not approach any single number.

Definition: A function f(x) is continuous at a point a if If f(x) is not continuous at x=a, we say that f(x) is discontinuous (or has a discontinuity) at a.

Example 2 Bakeco orders sugar from Sugarco. The per-pound purchase price of the sugar depends on the size of the order (see Table 1). Let x = number of pounds of sugar purchased by Bakeco f(x) = cost of ordering x pounds of sugar Then f(x) =25x for 0 ≤ x < 100 f(x) =20x for 100 ≤ x ≤ 200 f(x) =15x for x > 200 For all values of x, determine if x is continuous or discontinuous.

Example 2 – cont’d Solution It is clear that and do not exist. Thus, f(x) is discontinuous at x=100 and x=200 and is continuous for all other values of x satisfying x ≥ 0.

Taylor Series Expansion Higher Derivatives We define f(2)(a)=f ′′(a) to be the derivative of the function f ′(x) at x=a. Similarly, we can define (if it exists) f(n)(a) to be the derivative of f(n-1)(x) at x=a. Thus, for Example 3, f′′ (p) = 3000e-p(-1) – 3000e-p(1-p) Taylor Series Expansion In our study of queuing theory (Chapter 8), we will need the Taylor series expansion of a function f(x).

Given that fn-1(x) exists for every point on the interval [a,b], Taylor’s theorem allows us to write for any h satisfying 0 ≤ h ≤ b – a, (1) where (1) will hold for some number p between a and a + h. Equation (1) is the nth-order Taylor series expansion of the f(x) about a.

Example 4 Find the first-order Taylor series expansion of e-x about x = 0. Solution Since f′(x) = -e-x and f′′(x) = -e-x, we know that (1) will hold on any interval [0,b]. Also, f(0) = 1, f′(0) = -1, and f′′(x) = e-x. Then (1) yields the following first-order Taylor series expansion for e-x about x=0: This equation holds for some p between 0 and h.

Partial Derivatives We now consider a function f of n>1 variables (x1, x2, …,xn), using the notation f (x1, x2, …,xn) to denote such a function. Definition: The partial derivative of (x1, x2, …,xn) with respect to the variable xi is written , where

Suppose that for each i, we increase xi by a small amount Δxi Suppose that for each i, we increase xi by a small amount Δxi. Then the value of f will increase by approximately We will also use second-order partial derivatives extensively. We use the notation to denote a second-order partial derivative.

Review of Integral Calculus A knowledge of the basics of integral calculus is valuable when studying random variables. Consider two functions: f(x) and F(x). If F′(x), we say that F(x) is the indefinite integral of f(x). The fact that F(x) is the indefinite integral of f(x) is written The definite integral of f(x) from x=a to x=b is written

The Fundamental Theorem of Calculus states that if f(x) is continuous fro all x satisfying a ≤ x ≤ b, then where F(x) is any indefinite integral of f(x).

12.2 Differentiation of Integrals In order to differentiate a function whose value depends on an integral you should let f(x, y) be a function of variables x and y, and let g(y) and h(y) be functions of y. Then is a function only of y. Leibniz’s rule for differentiating an integral states that

12.3 Basic Rules of Probability Definition: Any situation where the outcome is uncertain is called an experiment. Definition: For any experiment, the sample space S of the experiment consists of all possible outcomes for the experiment. Definition: An event E consists of any collection of points (set of outcomes) in the sample space. Definition: A collection of events E1, E2,…,En is said to be a mutually exclusive collection of events if for i ≠ j (i=1,2,…,n and j=1,2,…n), Ei and Ej have no points in common.

With each event E, we associate an event Ē With each event E, we associate an event Ē. Ē consists of the points in the sample space that are not in E. With each event E, we also associate a number P(E), which is the probability that event E will occur when we perform the experiment. The probabilities of events must satisfy the following rules of probability:

Rule 1 For any event E, P(E) ≥ 0. Rule 2 If E=S (that is, if E contains all points in the sample space), then P(E) = 1. Rule 3 If E1, E2,…,En is mutually exclusive collection of events, then Rule 4 P(Ē) = 1 – P(E)

Definition: Suppose events E1 and E2 both occur with positive probability. Events E1 and E2 are independent if and only if P(E2|E1)=P(E2)(or equivalently, P(E1|E2) = P(E1)).

12.4 Bayes’ Rule Generally speaking, n mutually exclusive states of the world (S1, S2,…, Sn) may occur. The states of the world are collectively exhaustive: S1, S2,…, Sn include all possibilities. Suppose a decision maker assigns a probability P(Si) to Si. P(Si) is the prior probability of Si. Suppose that for each possible outcome Oj and each possible state of the world Si, the decision maker knows P(Oj|Si), the likelihood of the outcome Oj given the state of the world Si.

Bayes’ rule combines prior probabilities and likelihoods with the experimental outcomes to determine a post-experimental probability, or posterior probability, for each state of the world. Bayes’ rule:

12.5 Random Variables, Mean, Variance, and Covariance Definition: A random variable is a function that associates a number with each point in an experiment’s sample space. We denote random variables by boldface capital letters (usually X, Y, or Z). Definition: A random variable is discrete if it can assume only discrete values x1, x2,…. A discrete random variable X is characterized by the fact that we know the probability that X = xi (written P(X=xi)).

P(X=xi) is the probability mass function (pmf) for the random variable X. Definition: The cumulative distribution function F(x) = P(X≤x). For a discrete random variable X,

Continuous Random Variables If, for some interval, the random variable X can assume all values on the interval, then X is a continuous random variable. Probability statements about a continuous random variable X require knowing X’s probability density function (pdf). The probability density function f(x) for a random variable X may be interpreted as follows: For Δ small,

Mean and Variance of a Random Variable The mean (or expected value) and variance are two important measures that are often used to summarize information contained in a random variable’s probability distribution. The mean of a random variable X (written E(X)) is a measure of central location for the random variable. Mean of a discrete random variable X.

Mean of a continuous random variable, For a function h(X) of a random variable X (such as X2 and eX), E[h(X)] may be computed as follows: If X is a discrete random variable, If X is a continuous random variable,

Variance of the discrete random variable X, Variance of a continuous random variable, X, Also, var X may be found from the relation For any random variable X, (var X)½ is the standard deviation of X (written σx).

Independent Random Variables Definition: Two random variables X and Y are independent if and only if for any two sets A and B, The definition of independence generalizes to situations where more than two random variables are of interest. Loosely speaking, a group of n random variables is independent if knowledge of the values of any subset of the random variables does not change our view of the distribution of any of the other random variables.

For two random variables X and Y, the covariance of X and Y (written (X,Y) is defined by

Mean, Variance, and Covariance for Sums of Random Variables From given random variables X1 and X2, we often create new random variables (c is a constant): cX1, X1+c, X1+X2. The following rules can be used to express the mean, variance, and covariance of these random variables in terms of E(X1), E(X2), varX1, varX2, and cov(X1,X2). E(cX1)=cE(X1) E(X1+c)=E(X1)+c E(X1+X2)=E(X1)+E(X2)

If X1 and X2 are independent variables, var cX1 = c2varX1 var(X1+c) = var X1 If X1 and X2 are independent variables, var(X1+X2) = varX1 + varX2 In general, var(X1+X2) = varX1 + varX2 + 2cov(X1+X2) For random variables X1, X2 ,… Xn, Finally, for constants a and b, cov(aX1, bX2)=ab cov(X1,X2)

12.6 The Normal Distribution Definition: A continuous random variable X has a normal distribution if for some µ and σ > 0, the random variable has the following density function:

Useful Properties of Normal Distributions Property 1: If X is N(µ,σ2), then cX is N(cµ,c2 σ2). Property 2: If X is N(µ,σ2), then X + c (for any constant c) is N(µ+c, σ2). Property 3: If X1 is N(µ1,σ12), X2 is N(µ2,σ22), and X1 and X2 are independent, then X1+X2 is N(µ1+µ2,σ12+ σ22).

Finding Normal Probabilities via Standardization If Z is a random variable that is N(0,1), the Z is said to be a standardized normal random variable. If X is N(µ,σ2), then (X- µ)/σ is N(0,1). Suppose X is (µ,σ2) and we want to find P(a ≤ X ≤ b). We use the following relations (this procedure is called standardization):

If X1, X2,…Xn are independent random variables, then for n sufficiently large (usually n>30), the random variable X = X1 + X2 +…+Xn may be closely approximated by a normal random variable X′ that has E(X′) = E(X1) + E(X2) +…+E(Xn) and varX′ = varX1 + varX2+…+varXn. This result is known as the Central Limit Theorem. When we say that X′ closely approximates X, we mean that P(a ≤ X ≤ b) is close to P(a ≤ X′ ≤ b).

1.7 Finding Normal Probabilities with Excel Probabilities involving a standard normal random variable can be determines with the EXCEL=NORMSDIST function. The S in NORMSDIST stands for standardized normal. For example, P(Z≤-1) can be found by entering the formula =NORMSDIST(-1). The EXCEL=NORMSDIST function can be used to determine a normal probability for any normal (not just a standard normal) random variable.

If X is N(µ,σ2) the entering formula =NORMSDIST(a,µ,σ,1) will return P(X≤a). The 1 ensures that EXCEL returns the cumulative normal probability. Changing the last argument to “0” causes EXCEL to return the height of the normal density function for X=a.

12.7 Z-Transforms Consider a discrete random variable X whose only possible values are nonnegative integers. For n=0,1,2,… let P(X=n)=an. We define (for |z|≤1) the z-transform of X (call it pxT(z)) to be