Page 2
Semester 1: Probability Theory
Theory of Probability: Axiomatic approach, types of Events, Conditional Probability, Addition and Multiplication theorems for two events, Bayes theorem
Theory of Probability
Axiomatic Approach
The axiomatic approach to probability is based on a set of axioms formulated by Andrey Kolmogorov. The three main axioms are: (1) The probability of an event is a non-negative number. (2) The probability of the entire sample space is equal to 1. (3) For any sequence of mutually exclusive events, the probability of their union equals the sum of their individual probabilities.
Types of Events
Events can be classified into several types: (1) Simple events consist of a single outcome. (2) Compound events contain two or more simple events. (3) Mutually exclusive events cannot occur simultaneously. (4) Independent events are those where the occurrence of one does not affect the other.
Conditional Probability
Conditional probability is defined as the probability of an event occurring given that another event has already occurred. It is denoted as P(A|B) and calculated using the formula P(A|B) = P(A ∩ B) / P(B), provided P(B) > 0.
Addition Theorem for Two Events
The addition theorem states that for two events A and B, the probability of either event A or B occurring is given by P(A ∪ B) = P(A) + P(B) - P(A ∩ B). This accounts for the overlap between the two events.
Multiplication Theorem for Two Events
The multiplication theorem is used to determine the probability of both events A and B occurring. For independent events, it states that P(A ∩ B) = P(A) * P(B). For dependent events, the formula is P(A ∩ B) = P(A) * P(B|A).
Bayes Theorem
Bayes theorem provides a way to update the probability of a hypothesis based on new evidence. It is expressed as P(H|E) = [P(E|H) * P(H)] / P(E), where H is the hypothesis and E is the evidence. This theorem is particularly useful in scenarios involving conditional probabilities.
Random variables and Distribution functions: Discrete and Continuous random variables, Probability mass function, Probability density function and their properties
Random variables and Distribution functions
Discrete Random Variables
Discrete random variables take a countable number of distinct values. Common examples include the number of heads in a series of coin tosses or the number of students in a classroom. The probability mass function (PMF) defines the probability of each possible value.
Continuous Random Variables
Continuous random variables can take an infinite number of values within a given range. Examples include the exact height of individuals or the time taken to complete a task. The probability density function (PDF) is used to describe the probabilities of continuous outcomes, where probabilities are interpreted over an interval.
Probability Mass Function (PMF)
The PMF is a function that gives the probability that a discrete random variable is equal to a specific value. It must satisfy two properties: (1) the sum of the probabilities for all possible values must equal one and (2) the probability for any individual value must be non-negative.
Probability Density Function (PDF)
The PDF is a function associated with continuous random variables. The area under the curve represented by the PDF for an interval gives the probability that the variable falls within that range. The PDF must satisfy the properties of being non-negative and that the integral over the entire space must be equal to one.
Properties of PMF and PDF
Both PMF and PDF have several important properties. For PMF, these include total probability summing to one and non-negativity. For PDF, the integral across the complete range must be one. Additionally, cumulative distribution functions (CDF) may be derived from both PMF and PDF to provide probabilities up to a certain value.
Two dimensional random variables: Joint probability mass function, Marginal and Conditional probability functions, Two dimensional distribution functions
Two dimensional random variables
Joint probability mass function
The joint probability mass function describes the probability of two discrete random variables occurring simultaneously. It is denoted as P(X = x, Y = y) where X and Y are the two random variables. The joint PMF must satisfy the following conditions: 1. Non-negativity: P(X = x, Y = y) >= 0 for all x, y. 2. Normalization: The sum of all joint probabilities must equal 1, i.e., ∑_{x,y} P(X = x, Y = y) = 1.
Marginal probability functions
Marginal probability functions provide the probabilities of each individual random variable without reference to the other. They can be derived from the joint PMF. For a discrete random variable X, the marginal PMF is given by P(X = x) = ∑_{y} P(X = x, Y = y). Similarly, for Y, it is P(Y = y) = ∑_{x} P(X = x, Y = y). Marginal probabilities allow the assessment of the distribution of each variable independently.
Conditional probability functions
Conditional probability functions express the probability of one variable given the other. The conditional PMF of Y given X is defined as P(Y = y | X = x) = P(X = x, Y = y) / P(X = x) assuming P(X = x) > 0. This is useful in identifying dependencies between random variables and in the context of Bayes' theorem.
Two dimensional distribution functions
The two-dimensional distribution function, or joint cumulative distribution function (CDF), is defined as F(x, y) = P(X ≤ x, Y ≤ y). It provides the probability that the random variable X is less than or equal to x and Y is less than or equal to y. The joint CDF can be used to calculate joint probabilities and is related to the PMF through differentiation.
Mathematical Expectations: Expected value of random variables, expected value of functions of random variables, Properties of Expectation, Variance, Covariance
Mathematical Expectations
Expected Value of Random Variables
The expected value, or mean, of a random variable is a measure of the center of its distribution. For a discrete random variable X, it is calculated as E(X) = Σ [x * P(X = x)], where the sum is over all possible values x. For a continuous random variable, E(X) = ∫ x * f(x) dx, where f(x) is the probability density function.
Expected Value of Functions of Random Variables
If g is a function of a random variable X, the expected value of g(X) can be found using E(g(X)) = Σ [g(x) * P(X = x)] for discrete variables or E(g(X)) = ∫ g(x) * f(x) dx for continuous variables. This concept allows us to evaluate the mean of transformed random variables.
Properties of Expectation
The expectation operator has several important properties: 1. Linearity: E(aX + bY) = aE(X) + bE(Y) for any constants a and b. 2. Non-negativity: If X is non-negative, then E(X) ≥ 0. 3. Additivity: For independent random variables, E(X + Y) = E(X) + E(Y).
Variance
Variance is a measure of the dispersion of a random variable X, defined as Var(X) = E[(X - E(X))^2]. It quantifies how much the values of X deviate from the expected value. Variance can also be calculated as Var(X) = E(X^2) - (E(X))^2.
Covariance
Covariance measures the degree to which two random variables X and Y change together. It is defined as Cov(X, Y) = E[(X - E(X))(Y - E(Y))]. A positive covariance indicates that X and Y tend to increase together, while a negative covariance indicates that as one increases, the other tends to decrease.
Generating functions: M.G.F, C.G.F, P.G.F, Characteristic Functions, Uniqueness theorem, Chebychev's Inequality
Generating functions
Moment Generating Function (M.G.F)
The M.G.F is defined as M_X(t) = E[e^(tX)], where E denotes expectation and X is a random variable. It helps to find the moments of the distribution by differentiating the M.G.F.
Cumulant Generating Function (C.G.F)
The C.G.F is the logarithm of the M.G.F, given as K_X(t) = log(M_X(t)). It provides the cumulants of the distribution and helps in understanding the distribution's characteristics.
Probability Generating Function (P.G.F)
The P.G.F is defined for discrete random variables and is given by G_X(s) = E[s^X]. It is useful in calculating probabilities of various outcomes in a discrete setting.
Characteristic Functions
Characteristic functions are defined as ϕ_X(t) = E[e^(itX)], where i is the imaginary unit. They are helpful in studying the properties of distributions, especially in limit theorems.
Uniqueness Theorem
The uniqueness theorem states that if two random variables have the same M.G.F, then they have the same distribution. This property aids in identification and comparison of distributions.
Chebyshev's Inequality
Chebyshev's Inequality states that for any random variable X with mean μ and standard deviation σ, the probability that X deviates from μ by more than k standard deviations is at most 1/k^2. It provides a bound on the spread of distributions.
