Statistics for Data Science II: Quiz 1 prep

Discrete random variables you must know cold

Every Quiz 1 question uses at least one of these. Know the PMF, the range, the mean and the variance without looking.

• Geometric counts trials up to and including the first success, so its range starts at 1. \(P(X>k)=(1-p)^k\) and \(P(X\le k)=1-(1-p)^k\).
• Hypergeometric = draws without replacement: m items from N, of which r are "successes". With replacement it would be Binomial(m, r/N).
• Uniform{1,…,6} (a fair die): mean 3.5, variance 35/12 ≈ 2.917.
• A function of one random variable \(Y=g(X)\): list every \(x\), compute \(g(x)\), and add up the probabilities of all \(x\) that give the same \(y\). The range of \(Y\) is the set of values that actually occur.

1. Joint PMF of two random variables

Write it as a table: one row per value of one variable, one column per value of the other. Any probability about \((X,Y)\) is a sum of cells: \(P(X+Y=3)\) is the sum of the cells with \(t_1+t_2=3\).

• Unknown entries (a, b, k): use "sum of all cells = 1" plus whatever extra condition is given (a marginal, a conditional, or \(a=b/2\)).
• Formula-type joint PMFs such as \(f_{XY}(i,j)=\frac{i+2j}{k}\) on a grid: fill the table, add it up, set it equal to 1 to get k.
• Counting experiments (cards, balls without replacement): \(f_{XY}(x,y)=\frac{\binom{a}{x}\binom{b}{y}\binom{\text{rest}}{n-x-y}}{\binom{N}{n}}\). The "rest" factor is the one people forget.

2. Marginal PMF

• The marginal is an ordinary PMF: it adds up to 1.
• Different joint PMFs can have the same marginals. The marginals alone never tell you the joint (unless the variables are independent).
• In the coin example: \(f_X=(1/2,1/2)\), \(f_Y=(1/4,1/2,1/4)\).

3. Conditional distribution

• A conditional PMF adds up to 1 over \(t_2\): \(\sum_{t_2} f_{Y\mid X=t_1}(t_2)=1\).
• Range of \((Y\mid X=t_1)\) = the values of \(t_2\) whose cell in that row is non-zero. It can be smaller than \(T_Y\).
• Factorisation: \(f_{XY}(t_1,t_2)=f_{Y\mid X=t_1}(t_2)\,f_X(t_1)=f_{X\mid Y=t_2}(t_1)\,f_Y(t_2)\). Use it to build a joint PMF from a two-stage experiment.
• Conditioning on an event works the same way: \(P(X=t\mid A)=P(X=t, A)/P(A)\).

4. Poisson splitting (the most repeated idea in Week 1)

• \(f_{NX}(n,k)=0\) when \(k>n\) (you cannot have more spam emails than emails).
• Joint probability of "exactly b boys and g girls": \(P(B=b,G=g)=e^{-\lambda}\frac{\lambda^{b+g}}{b!\,g!}p^g(1-p)^b\). With p = 0.5 this is \(\frac{e^{-\lambda}(\lambda/2)^{b+g}}{b!\,g!}\).
• \(E[X]=\lambda p\) (useful in Week 3 questions).

5. Three or more random variables

• Joint PMF \(f_{X_1X_2X_3}(t_1,t_2,t_3)=P(X_1=t_1,X_2=t_2,X_3=t_3)\), usually given as a list of rows (t₁, t₂, t₃, prob).
• Marginalise by summing out the variables you do not want: \(f_{X_1X_2}(t_1,t_2)=\sum_{t_3}f_{X_1X_2X_3}(t_1,t_2,t_3)\); \(f_{X_1}(t_1)=\sum_{t_2,t_3}f(t_1,t_2,t_3)\).
• Condition on any subset: \(f_{X_1\mid X_2=a,X_3=b}(t)=\dfrac{f_{X_1X_2X_3}(t,a,b)}{f_{X_2X_3}(a,b)}\); \(P(X_3=1,X_4=1\mid X_1=1,X_2=1)=\dfrac{f(1,1,1,1)}{f_{X_1X_2}(1,1)}\).
• Chain rule (any order): \(f_{X_1X_2X_3}=f_{X_1\mid X_2,X_3}\cdot f_{X_2\mid X_3}\cdot f_{X_3}\). Typical quiz item: given three of these four numbers, find the fourth.
• \(f_{XYZ}=f_{X\mid Y,Z}\,f_{YZ}\) is always true; \(f_{X\mid Y,Z}\,f_Y\,f_Z\) is true only under independence of Y and Z.

1. Independence of two random variables

• One failing cell is enough to prove dependence. One matching cell proves nothing.
• Zero test: if some cell is 0 while its row total and column total are both non-zero, \(X\) and \(Y\) are dependent. Fastest check in an exam.
• Row test: independent ⇔ every row of the table is a multiple of every other row (all rows proportional to the column totals).
• Joint PMF that factors as \(g(x)h(y)\) on a rectangular range (e.g. \(\frac{9}{16\cdot 4^{x+y}}=\frac{3}{4^{x+1}}\cdot\frac{3}{4^{y+1}}\)) ⇒ independent. If the range is not a rectangle (e.g. \(x\lt y\)) they are dependent.
• Two dice: \(X\) = die A and \(Y\) = number of sixes are dependent (\(X=6\) forces \(Y\ge1\)). Heads and tails in the same tosses are dependent (\(Y=n-X\)).

2. Many random variables, i.i.d.

• \(X_1,\dots,X_n\) are independent if the joint PMF is the product of all n marginals, for every combination of values. Then every subset is independent too, and conditioning on some of them does not change the others: \(f_{X_1X_2\mid X_3=t}(a,b)=f_{X_1}(a)f_{X_2}(b)\).
• Pairwise independence does not imply independence of all three (even-parity example: \(X_3=X_1\oplus X_2\)).
• i.i.d. = independent and identically distributed (same PMF). Repeated tosses, die throws, samples.
• "All n samples exceed c": \(P(X_1>c,\dots,X_n>c)=P(X>c)^n\). For 10 i.i.d. Geometric(0.2), \(P(\text{all}>10)=(0.8^{10})^{10}=0.8^{100}\).
• "Exactly one sample equals a and exactly one equals b" among n i.i.d.: choose positions, \(n(n-1)\,P(a)P(b)\,[1-P(a)-P(b)]^{n-2}\).

3. Functions of random variables

• \(Z=g(X_1,\dots,X_n)\): for each value z, \(f_Z(z)=\sum f_{X_1\dots X_n}(t_1,\dots,t_n)\) over all tuples with \(g(t_1,\dots,t_n)=z\).
• Work it as a table: list each tuple with its probability and its g-value, then group by g-value. The range is the set of g-values with non-zero probability.
• Joint PMF of two functions \((g,h)\): group by the pair of values. Same method.
• One-to-one functions keep the probabilities and just relabel; many-to-one functions merge probabilities.

4. Sum of two random variables

• Poisson + Poisson: independent \(X\sim\)Poisson(λ₁), \(Y\sim\)Poisson(λ₂) ⇒ \(X+Y\sim\)Poisson(λ₁+λ₂), and \((X\mid X+Y=n)\sim\) Binomial\((n,\frac{\lambda_1}{\lambda_1+\lambda_2})\).
• Binomial + Binomial (same p): \(X_1+X_2\sim\)Binomial(n₁+n₂, p), and \((X_1\mid X_1+X_2=m)\sim\) HyperGeo(n₁+n₂, n₁, m), which does not depend on p.
• Sum of n i.i.d. Bernoulli(p) = Binomial(n, p). Given the sum is k, each \(X_i\) is 1 with probability k/n: \(P(X_1=0\mid Y=8)=2/10\) when n = 10.
• Two i.i.d. Geometric(p): \(P(X+Y=n)=(n-1)p^2(1-p)^{n-2}\) for n ≥ 2, and \((X\mid X+Y=n)\) is uniform on {1, …, n−1}, whatever p is. So \(P(X=6\mid X+Y=10)=1/9\).
• Difference of i.i.d. Geometric(p): \(P(X-Y=k)=\frac{p(1-p)^{k}}{2-p}\) for k ≥ 1 (and \(\frac{p}{2-p}\) for k = 0).
• Two fair dice: \(P(X+Y=s)=\frac{6-|s-7|}{36}\); \(P(X-Y=d)=\frac{6-|d|}{36}\).

5. Maximum and minimum

• PMF of the max (integer values): \(P(\max=k)=F(k)^n-F(k-1)^n\). Two dice: \(P(\max=k)=\frac{k^2-(k-1)^2}{36}=\frac{2k-1}{36}\).
• PMF of the min: \(P(\min=k)=P(\min\ge k)-P(\min\ge k+1)\). Two dice: \(P(\min=k)=\frac{(7-k)^2-(6-k)^2}{36}=\frac{13-2k}{36}\).
• Min of independent Geometrics is Geometric: \(\min(X,Y)\sim\) Geometric\((1-(1-p_1)(1-p_2))\).
• Max of Bernoullis: \(P(\max=0)=\prod P(X_i=0)\). Min of Bernoullis: \(P(\min=1)=\prod P(X_i=1)\).
• i.i.d. Uniform{1,…,m}: \(P(\min(X,Y)=k)=\frac{(m-k+1)^2-(m-k)^2}{m^2}\).

1. Expected value

• Linearity (always true, even for dependent variables): \(E[aX+bY+c]=aE[X]+bE[Y]+c\).
• \(E[XY]=E[X]E[Y]\) if X and Y are independent. The converse is false.
• \(X\ge0\) ⇒ \(E[X]\ge0\). A constant has \(E[c]=c\).
• \(E[X^2]\ne(E[X])^2\) in general (the gap is the variance).

• Lottery / coupon expected prize: buying 2 of 600 tickets where the prizes total ₹1800: each ticket's mean prize is 1800/600 = 3, so \(E=2\times3=6\), i.e. ₹6. Without-replacement draws still add means.
• Random number of customers: if \(N\sim\)Poisson(λ) customers each spend an average of m, expected total = λ·m. Expected buyers when each buys with prob p: λp.
• Investments: a ₹2000 share going ×1.5 w.p. 0.3 or ×0.5 w.p. 0.7 has expected value 2000(0.45+0.35) = 1600. Independent day-to-day factors multiply their means.

2. Variance and standard deviation

• \(\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)\), \(\mathrm{SD}(aX+b)=|a|\,\mathrm{SD}(X)\). Adding a constant changes nothing.
• \(\mathrm{Var}(X\pm Y)=\mathrm{Var}X+\mathrm{Var}Y\pm2\,\mathrm{Cov}(X,Y)\). If independent the covariance term is 0, so \(\mathrm{Var}(X-Y)=\mathrm{Var}X+\mathrm{Var}Y\) (variances add, never subtract).
• Independent \(X, Y\): \(\mathrm{Var}(XY)=E[X^2]E[Y^2]-(E[X]E[Y])^2\), where \(E[X^2]=\mathrm{Var}X+(E X)^2\).
• \(\mathrm{Var}(X)=\tfrac12E[(X-X')^2]\) for an i.i.d. copy \(X'\).
• Standardised variable \(Z=\frac{X-\mu}{\sigma}\) has mean 0 and variance 1.

3. Covariance

• \(\mathrm{Cov}(X,X)=\mathrm{Var}(X)\); \(\mathrm{Cov}(X,c)=0\); symmetric.
• Bilinear: \(\mathrm{Cov}(aX+bY,\,cZ+dW)=ac\,\mathrm{Cov}(X,Z)+ad\,\mathrm{Cov}(X,W)+bc\,\mathrm{Cov}(Y,Z)+bd\,\mathrm{Cov}(Y,W)\). Constants added inside disappear.
• Independent ⇒ Cov = 0. Cov = 0 does not imply independent (e.g. X uniform on {−1, 0, 1} and Y = X²).
• \(\mathrm{Cov}(X,X+Y)=\mathrm{Var}X+\mathrm{Cov}(X,Y)\).

4. Correlation coefficient

• \(\rho=\pm1\) exactly when \(Y=aX+b\) (sign of a). Two balls from 5 white + 5 red: X red + Y white = 2, so \(\rho(X,Y)=-1\).
• \(\rho(aX+b,\,cY+d)=\pm\rho(X,Y)\), sign of ac. Scaling never changes the size of ρ.
• ρ is the covariance of the standardised variables: \(\mathrm{Cov}\!\big(\frac{X-\mu_X}{\sigma_X},\frac{Y-\mu_Y}{\sigma_Y}\big)=\rho\). If X and Y are independent this is 0.
• Joint PMF that factors (e.g. \(\frac{(x+1)(y+1)}{27}\)) ⇒ independent ⇒ ρ = 0.

5. Bounds from mean and variance

• Markov with a strict inequality on an integer variable: \(P(X\gt 50)=P(X\ge51)\le\frac{\mu}{51}\).
• Chebyshev needs a symmetric interval around μ. "Between 5 and 15" with μ = 10 means \(|X-10|\lt5\). If the interval is not symmetric, use the half-width of the largest symmetric interval inside it.
• "At least 130 or at most 70 heads in 200 fair tosses": μ = 100, σ² = 50, \(P(|X-100|\ge30)\le50/900=1/18\).
• Chebyshev with k ≤ 1 gives a useless bound (≥ 1). The quiz still asks for it: answer 1.
• Find n so that a Chebyshev bound on \(P(|\bar X-\mu|\ge\varepsilon)\) is below δ: \(\frac{\sigma^2}{n\varepsilon^2}\lt\delta\) ⇒ \(n\gt\frac{\sigma^2}{\varepsilon^2\delta}\).

Week 4 opens on Seek around 23 Oct. These notes follow the Week 4 slides (already in the course's slides folder) and the Jun 2026 run's Week 4 lectures and assignments. Both recent Quiz 1 papers had two to four Week 4 items worth 6–8 marks.

1. Cumulative distribution function (CDF)

• \(P(a\lt X\le b)=F(b)-F(a)\). Learn this exact form: left end open, right end closed.
• \(P(X\gt a)=1-F(a)\); \(P(X=a)\) = size of the jump of F at a; \(P(X\lt a)=F(a^-)\) (the value just before a).
• Discrete X: F is a staircase. The jump points are the range of X and the jump sizes are the PMF. Between jumps F is flat, so \(P(X\le 1.5)=F(1)\) if nothing lies in (1, 1.5].
• Geometric(p): \(F(x)=1-(1-p)^k\) for \(k\le x\lt k+1\).
• For \(Y=\frac{X-\mu}{\sigma}\): \(F_Y(y)=F_X(\mu+y\sigma)\).

2. Continuous and mixed random variables

• A CDF with both slopes and jumps is a mixed random variable: the jumps carry point probabilities. E.g. \(F(x)=0.5+0.1x\) on [0, 5] has \(P(X=0)=0.5\).
• "Which cannot be the CDF of a continuous random variable?" → look for a jump (a break where the two pieces do not meet) or a CDF that does not reach 1.
• Continuous models: heights, weights, waiting times, rainfall, stock prices. They approximate discrete variables with huge ranges (e.g. Binomial with large n).

3. Probability density function (PDF)

• Valid density: \(f(x)\ge0\) everywhere and \(\int_{-\infty}^{\infty}f(x)\,dx=1\) (and piecewise continuous). \(\frac4{15}x^3\) on [−1, 2] fails because it is negative for x < 0.
• A PDF value can be bigger than 1. It is a density, not a probability.
• \(P(x_0-\varepsilon\lt X\lt x_0+\varepsilon)\approx 2\varepsilon f(x_0)\): a higher density means values near \(x_0\) are more likely. "\(f(2)=2f(1)\) ⇒ probability near 2 ≈ twice that near 1" is true only approximately.
• Support = \(\{x: f(x)\gt0\}\).
• Finding k: integrate the PDF over its support and set it to 1. \(k(6-x)\) on (0, 6): \(k\cdot18=1\Rightarrow k=\frac1{18}\).
• Median / percentile: solve \(F(m)=0.5\) (or 0.75 for the 75th percentile). For \(4x^3\) on [0, 1], \(F=x^4\), median \(=0.5^{1/4}\approx0.841\).
• Conditional on an event: \(P(X\gt2\mid X\gt1)=\frac{P(X\gt2)}{P(X\gt1)}\), since {X > 2} lies inside {X > 1}.
• Integrals you need: \(\int x^n dx=\frac{x^{n+1}}{n+1}\), \(\int e^{-ax}dx=-\frac{e^{-ax}}{a}\). Areas of triangles and trapezoids often replace integrals for straight-line PDFs.

4. Common continuous distributions

• Uniform: probability = length of the overlap ÷ (b − a). Conditioning a uniform on a sub-interval gives a uniform on that sub-interval: \(P(X\lt10\mid X\lt20)=\frac{10}{20}\) for Uniform[0, 50].
• Exponential: \(P(X\gt x)=e^{-\lambda x}\), \(P(a\lt X\lt b)=e^{-\lambda a}-e^{-\lambda b}\). "Mean 10 minutes" means λ = 0.1. Memoryless: \(P(X\gt s+t\mid X\gt s)=P(X\gt t)\).
• Normal: bell curve centred at μ (the most likely region), symmetric. Standardise: \(P(X\le x)=F_Z\!\big(\frac{x-\mu}{\sigma}\big)\). Symmetry: \(F_Z(-z)=1-F_Z(z)\). Normal(2, 5) means variance 5, so σ = √5.
• Rules of thumb: \(P(|Z|\le1)\approx0.683\), \(P(|Z|\le2)\approx0.954\), \(P(|Z|\le3)\approx0.997\); \(F_Z(1)\approx0.841\), \(F_Z(-1)\approx0.159\).
• "Replace at most 15.8% under warranty" with lifetime Normal(μ, σ²) and \(F_Z(-1)=0.158\) ⇒ warranty = μ − σ.