Page 8
Semester 3: Probability Theory
Random Events and Random Variables - Probability axioms - Bayes Theorem
Random Events and Random Variables
Random Events
Random events are outcomes that cannot be predicted with certainty, represented as elements within a sample space. Examples include rolling a die or flipping a coin.
Random Variables
Random variables are numerical outcomes of random events and can be classified into discrete and continuous variables. Discrete variables take specific values, while continuous variables can take any value within a range.
Probability Axioms
Probability is defined through three fundamental axioms: 1. Non-negativity: The probability of any event is greater than or equal to zero. 2. Normalization: The probability of the entire sample space equals one. 3. Additivity: For mutually exclusive events, the probability of their union is the sum of their individual probabilities.
Bayes Theorem
Bayes Theorem relates conditional probabilities and is expressed as P(A|B) = P(B|A) * P(A) / P(B). It allows for the updating of probabilities based on new evidence and plays a crucial role in statistics and decision-making.
Parameters of the Distribution - Expectation, Moments, Chebyshev Inequality
Parameters of the Distribution
Expectation
Expectation, also known as the mean, is a measure of the central tendency of a random variable. It gives the average value that the variable takes over numerous trials. For a discrete random variable X, the expectation E(X) is calculated as E(X) = Σ [x * P(X=x)], where x runs over all possible values of X. For a continuous random variable, the expectation is given by E(X) = ∫ x * f(x) dx, where f(x) is the probability density function.
Moments
Moments are quantitative measures related to the shape of a distribution. The n-th moment of a random variable is defined as E(X^n). The first moment is the expectation. The second moment is related to variance, calculated as E(X^2) - [E(X)]^2. Higher-order moments can provide insights into skewness and kurtosis, which describe the asymmetry and peakedness of the distribution, respectively.
Chebyshev's Inequality
Chebyshev's Inequality is a fundamental result in probability theory. It provides a bound on how much of the probability distribution is within a certain number of standard deviations from the mean. Specifically, for any k > 1, Pr(|X - μ| >= kσ) <= 1/k^2, where μ is the mean, σ is the standard deviation, and Pr represents the probability. This inequality is useful because it holds for any distribution with a finite mean and variance.
Characteristic functions - Properties and applications
Characteristic functions - Properties and applications
Definition of Characteristic Functions
Characteristic functions are a type of Fourier transform used in probability theory to describe the distribution of a random variable. For a random variable X, the characteristic function is defined as phi(t) = E[e^(itX)], where E denotes the expected value and i is the imaginary unit.
Properties of Characteristic Functions
Applications of Characteristic Functions
Some Probability distributions - Binomial, Poisson, Gamma, Beta, Cauchy
Some Probability Distributions
Binomial Distribution
The Binomial distribution describes the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success. It is defined by two parameters: n (the number of trials) and p (the probability of success). The probability mass function is given by P(X=k) = (n choose k) * p^k * (1-p)^(n-k), where k is the number of successes.
Poisson Distribution
The Poisson distribution models the number of events occurring within a fixed interval of time or space when these events occur with a known constant mean rate and independently of the time since the last event. It is characterized by the parameter λ (lambda), which is the average number of events in the given interval. The probability mass function is given by P(X=k) = (e^(-λ) * λ^k) / k!, where k is the number of events.
Gamma Distribution
The Gamma distribution is a two-parameter family of continuous probability distributions. It is defined by shape parameter k and scale parameter θ. It is often used to model waiting times and is particularly useful in queuing models. The probability density function is given by f(x; k, θ) = (1 / (θ^k * Γ(k))) * x^(k-1) * e^(-x/θ) for x > 0, where Γ(k) is the gamma function.
Beta Distribution
The Beta distribution is a family of continuous distributions defined on the interval [0, 1] parameterized by two positive shape parameters, α and β. It is useful in modeling random variables limited to intervals of finite length. The probability density function is given by f(x; α, β) = (1 / B(α, β)) * x^(α-1) * (1-x)^(β-1) for 0 < x < 1, where B(α, β) is the beta function.
Cauchy Distribution
The Cauchy distribution is a continuous probability distribution that is defined by its location parameter x₀ and scale parameter γ. It is characterized by its peakedness and heavy tails, making it different from the normal distribution. The probability density function is given by f(x; x₀, γ) = (1 / (πγ)) * [γ^2 / ((x - x₀)^2 + γ^2)], where x can take any real value.
Limit Theorems - Law of Large Numbers - Central Limit Theorem - Strong Law
Limit Theorems
Law of Large Numbers
The Law of Large Numbers states that as the number of trials increases, the sample average will converge to the expected value. This principle ensures that empirical probabilities stabilize with large samples. There are two versions: the weak law and the strong law.
Weak Law of Large Numbers
The weak law states that for any positive epsilon, the probability that the sample mean deviates from the expected mean by more than epsilon approaches zero as the sample size goes to infinity. This focuses on convergence in probability.
Strong Law of Large Numbers
The strong law provides a stronger assertion by stating that the sample averages almost surely converge to the expected value as the sample size tends to infinity. This guarantees that the convergence does indeed happen with probability one.
Central Limit Theorem
The Central Limit Theorem indicates that the distribution of the sum of a large number of independent, identically distributed variables approaches a normal distribution, regardless of the original distribution. This is crucial for statistical inference as it justifies the use of the normal distribution in hypothesis testing.
Applications of Central Limit Theorem
The Central Limit Theorem is applied in various fields such as finance, genetics, and quality control to derive predictions and perform statistical analyses, allowing for the approximation of probabilities and confidence intervals.
