Table of Contents
| Table of Contents | ||||
|---|---|---|---|---|
|
Assigned Reading
Although our text does not conduct a formal probability review, it would be advised that you consult your prerequisite course's probability textbook. The free, online Khan Academy also has a nice collection of short probability and statistics tutorials covering a wide variety to topics applicable to this course.
Useful Equations
Rules of Integration
Chain Rule for Integration
| Mathinline | ||
|---|---|---|
|
If
| Mathinline | ||
|---|---|---|
|
| Mathinline | ||
|---|---|---|
|
...
Rules of Differentiation
Chain Rule for Differentiation
Laws for Probabilities of Events
Venn Diagram
Law of Addition
Definition of Conditional Probability
Bayes' Rule
Random Variables (RVs)
Conditional Probability
If X and Y are discrete RVs, the definition of conditional probability for all values of y such that Pr(Y = y) > 0 is
If X and Y are continuous RVs and have joint PDF fXY(x,y), then the conditional PDF for all values of y such that fY(y) > 0 is
For RV X, we can find Pr(X <= x) by conditioning on RV Y. If Y is a discrete RV, then
If Y is a continuous RV with desity fY(y), then
Conditional Expectation
If X and Y are jointly discrete RVs, then for all values of y, such that Pr(Y = y) > 0,
Likewise, if X and Y are jointly continuous RVs, then for all values of y such that fY(y) > 0,
Computing Expectations by Conditioning
For RV X, we can find E[X] by conditioning on RV Y. If Y is a discrete RV, then
If Y is a continuous RV with density fY(y), then
Integer Series Identities
If 0 < p < 1, then
If a ≠ 1, then
If a ≠ 1, then
where, k factorial = k! = (k)(k - 1)(k - 2)(...)(1).
Combinations of Choosing k from n Elements
Frequently Used Random Variables
Exponential Continuous Random Variable
RV X with parameter α parameter α > 0.Supports that supports [0, ∞)
Memoryless property:
Uniform Continuous Random Variable
RV X.Supports that supports [a, b]
Bernoulli Discrete Random Variable
RV X with parameter 0 < p < 1.Supports k that supports k = {0, 1}
Binomial Discrete Random Variable
RV X with parameters positive integer n > 0, and 0 < p < 1.Supports k that supports k = {0, 1, ..., n}
Geometric Discrete Random Variable
RV X with parameter 0 < p < 1 indicating the probability of success.
The number X of Bernoilli trials needed to get one success, supported on the set k = {1, 2, ...}.
Memoryless property:
The number of Y = X - 1 failures before the first success, supported on the set k = {0, 1, ...}.
Poisson Discrete Random Variable
RV X with parameter λ > 0 that supports k = {0, 1, ...Supports k = {0, 1, ...} }
Review of Random Variables (RVs)
Cumulative Distribution Function
A RV X is defined by a function FX(x), called a cumulative distribution function (CDF) of X, which has the following properties:
Example
CDF of a RV:
Probability
For a RV X and any value a, FX(a) represents the probability that X is less than or equal to a, i.e.:
Probability Space
FX(x) defines a probability space, and
Continuous RVs
If the CDF of FX(x) of the RV X is a continuous function of x, then X is a continuous RV.
Example
CDF of a continuous RV:
CDF of a non-continous RV:
Probability Density Function
If a RV X is continuous, and there is a function fX(x), called the probability density function (PDF), such that:
then X is an absolutely continuous RV.
If X is differentiable at x, then:
The following are properties of a PDF:
Example
PDF of a continuous RV that is uniformly distributed on the interval [0,C]:
PDF of a continuous RV that is exponentially distributed with parameter α > 0:
Discrete Random Variables
If the CDF FX(x) of a RV X is constant between any two consecutive values in a discrete set of increasing values of x, X is referred to as a discrete random variable.
Example
CDF of a discrete RV:
Probability Mass Function
If a RV X is discrete, and FX(x), has step increases at the points {..., x-1, x0, x1, ...}, then:
If there is a smallest value xj at which a step increase occurs in FX(x), then:
piX is referred to as the probability mass function (PMF) of X. {..., x-1, x0, x1, ...} is referred to as the support of X. Note that piX atx = xi is often referred to as pX(xi).
The following are properties of a PMF:
Example
A discrete RV X that is binomially distributed with parameters N and 0 <= p <= 1 with support x0 = 0, x1 = 1, ..., xN = N and a PMF:
Expected Value
The expected value (i.e. mean or statistical average) of a RV X is given by:
Autocorrelation
The autocorrelation of a RV X is given by:
nth Moment
The nth moment of a RV X is given by:
Variance
The variance of a RV X is given by:
Standard Deviation
The standard deviation of a RV X is given by:
Expected Value of a Function
The expected value of a function of a RV X is given by:
Examples of Mean, Variance, Standard Deviation, etc.
Continuous RVs
Consider the RV X with uniform distribution over [0,1]:
Suppose h(x) = x3/2:
Discrete RVs
Consider the RV X with binomial distribution and parameters N = 2 and p = 0.2. Support of X is {x0 = 0, x1 = 1, x2 = 2}.
Suppose h(x) = x1/2:
Joint Characterization of RVs
Jointly Continuous RVs
Two jointly (absolutely) continuous RVs X and Y are defined by a joint PDF:
and a joint CDF:
Jointly continuous RVs have the following properties:
Example
Consider RVs X and Y with a joint PDF:
Find Pr(X - Y < 1):
Jointly Discrete RVs
Two jointly discrete RVs X and Y are characterized by a support:
a joint PMF {pi,j} for all i and j, and a joint CDF given by:
with the following properties:
Marginal PDFs
The PDFs of the individual absolutely continuous RVs, or marginal PDFs, are determined from the joint PDF by:
Marginal PMFs
The PMFs of the individual absolutely discrete RVs, or marginal PMFs, are determined from the joint PMF by:
Shorthand Notation
If no ambiguity results, subscripts or superscripts for the PDF, PMF, or CDF are often omitted. For example:
Cross-Correlation
Continuous RVs
The cross-correlation of two jointly absolutely continuous RVs X and Y is given by:
Discrete RVs
The cross-correlation of two jointly discrete RVs X and Y is given by:
Covariance
The covariance of two RVs X and Y is given by:
If the Cov(X,Y) = 0, X and Y are uncorrelated.
Independence
Continuous RVs
Two jointly absolutely continuous RVs X and Y are independent if:
Or equivalently:
Discrete RVs
Two jointly discrete RVs X and Y are independent if:
Or equivalently:
Conditional Probability
Continuous RVs
Conditional PDF
If X and Y are jointly absolutely continuous RVs, the conditional PDF of X given Y is defined as:
If fY(y) ≠ 0:
So, fX|Y(y) is a valid PDF for the RV X given Y = y. This is written as X | Y = y.
Conditional CDF
If X and Y are jointly absolutely continuous RVs and fY(y) ≠ 0, the conditional CDF of X given Y = y is defined as:
If X and Y are jointly absolutely continuous RVs and Pr(Y in the set B) ≠ 0, the conditional CDF of X given Y in the set B is defined as:
Examples
Example 1
B = (-∞, a], then Y in the set B ↔ Y ≤ a.
Example 2
B = [0,1] U [3,4], then Y in the set B ↔ (0 ≤ Y ≤ 1 or 3 ≤ Y ≤ 4).
Example 3
For 0 ≤ y ≤ 1:
For y < 0 or y > 1, fX|Y(x|y) = 0 and FX|Y(x|y) is not defined.
Discrete RVs
Conditional PMF
Suppose X and Y are jointly discrete RVs with support {(xi,yj)}, i and j integers. The conditional PMF of X given Y is defined as:
If pjY != 0:
So, pi|jX|Y, i = ..., -1, 0, 1, ... is a valid PMF for the RV X given Y = yj, with support {..., x-1, x0, x1, ...}.
Conditional CDF
If X and Y are jointly discrete RVs and pjY != 0, the conditional CDF of X given Y = yj is defined as:
If X and Y are jointly discrete RVs and Pr(Y in the set B) != 0, the conditional CDF of X given Y in the set B is defined as:
