Forem: Yadav Prasad G B

Discrete Random Probability

Yadav Prasad G B — Sun, 19 Oct 2025 16:36:56 +0000

Random Variable

An assignment of a value to every possible outcome.
A function from sample space to the real numbers. It is of two types: discrete and continuous random variables.
It is denoted by X, and its numeric value is denoted by x.

For Example:

Tossing a coin once may land either Heads or Tails :

S = {H, T}

A random variable ( X ) can be defined as:

(X = 1) if it is Heads
(X = 0) if it is Tails

We can also have more than one random variable in an experiment. The diagram below takes a random data point and maps it to the height scale (in inches). We can also map its weight (in kg).

Probability Mass Function (PMF) :

Probability Mass Function is a function that gives the probability that a discrete random variable takes a specific value. It is denoted by P(X = x) probability that X takes the value x.

Properties of PMF:
0≤P(X=x)≤1 for all possible values x.
The sum of probabilities for all possible values of x : ∑x P(X=x)=1.

we use PMF to generally study how probabilities are distributed across all possible outcomes of a discrete random variable.

For Example : A die is rolled and the following sample space are {1, 2, 3, 4, 5, 6} assume that the random variable X is equals to the values in the sample space so the PMF for this experiment would be :

P(X = 1) = 1/6 P(X = 2) = 1/6 P(X = 3) = 1/6
P(X = 4) = 1/6 P(X = 5) = 1/6 P(X = 6) = 1/6

Binomial PMF

Binomial PMF is used to determine the probability of k successes in n independent events where each trial has only two outcomes.

if we take the coin toss example we can assign some probability value p if heads lands else 1 - p if tails lands and taking X = number of success in n trails.

the formula for this is given by,

P(X=k)= n!/(k!(n-k)!) * p^k(1−p)^n−k

why bothering finding the combination of k different sequences of occurring heads ? Let us take an example of finding the P(X = 2) given the number of coin tosses are 4.

n = 4, k = 2, X = p if heads appears else (1-p) if tails appears,

P(X = 2) = P(HHTT) + P(HTHT) + P(HTTH) + P(THHT) + P(THTH) + P(TTHH) = 6 * p^2 * (1-p)^2 which is equal to 4!/2!*(4-2)! * p^2 * (1-p)^2

Expectation

It is the average values of a random variable that you will expect when performing an experiment repeatedly.

it is denoted by E[X], generally finds the center of gravity of PMF

If a random variable X can take values x1,x2,x3,...x with probabilities P(X=xi) then the expected value is:
E[X]=∑xi P(X=xi)

Let us take an example, You are playing a Game where the chances of winning $1 is 1/6 and chances of winning $2 is 1/2 and chances of winning $4 is 1/3, so much the player should bet in-order to break even in the long term ??

E[X] = 1/6 * 1 + 1/3 * 2 + 1/3 * 4 = 2.5

therefore the player can bet upto $2.5 per game to break even in the long term if the player bets more than the expected value then he's more likely to be in loss after n games.

But how does expectation determines the center of gravity of a PMF, lets see this visually

Properties of Expectations

Let X and Y be random variables, and a, b, c be constants.

1. Linearity of Expectation
E[aX + bY + c] = a E[X] + b E[Y] + c

Expectation distributes over addition and scalar multiplication. This holds even if X and Y are not independent.

Example:

If E[X] = 2 and E[Y] = 3,

E[2X + 5Y + 1] = 2·2 + 5·3 + 1 = 22

2. Expectation of a Constant
E[c] = c

A constant always takes the same value, so its expected value is itself.

3. Expectation of a Sum
E[X + Y] = E[X] + E[Y]

A special case of linearity: the mean of the sum equals the sum of the means.

4. Independent Random Variables
If X and Y are independent:

E[XY] = E[X] · E[Y]

5. Non-Negativity
If X ≥ 0, then E[X] ≥ 0.

6. Function of a Random Variable
For any function g(X):

E[g(X)] = Σ g(x) · P(X = x)

Example

Let X be the number shown on a fair die: X ∈ {1, 2, 3, 4, 5, 6}

E[X] = ΣPX(x) . x = (1/6) (1 + 2 + 3 + 4 + 5 + 6)/6 = 3.5

Now if Y = 2X + 1:

E[Y] = 2 E[X] + 1 = 2·3.5 + 1 = 8

Variance Properties for Discrete Random Variables (PMF)

Variance measures how much the values of a random variable differ from the expected mean. It tells us the spread or dispersion of data around the mean.And the square root of the variance is called the standard deviation.

The formula for discrete random variable is given by,
Var(X) = Σ (xᵢ - E[X])² · P(X = xᵢ)

Properties of Variance

Let X and Y be discrete random variables and a, b, c be constants.

1. Variance of a Constant

Var(c) = 0

A constant does not vary.
Example: If X = 5 always, then Var(X) = 0.

2. Scaling Property

Var(aX) = a^2 * Var(X)

Multiplying a random variable by a constant scales variance by a^2.
Example: If Var(X) = 2 and Y = 3X, then Var(Y) = 3^2 * 2 = 18.

3. Adding a Constant

Var(X + b) = Var(X)

Adding a constant shifts the distribution but does not change spread.
Example: Var(X) = 2, then Var(X + 5) = 2.

4. Variance of a Sum (Independent Random Variables)

Var(X + Y) = Var(X) + Var(Y) (if X and Y are independent)

For independent variables, total variance is the sum of individual variances.
If not independent: Var(X + Y) = Var(X) + Var(Y) + 2 * Cov(X,Y)

5. Variance of a Linear Combination

Var(aX + bY) = a^2 * Var(X) + b^2 * Var(Y) + 2ab * Cov(X,Y)

If X and Y are independent, Cov(X,Y) = 0.

6. Non-Negativity

Var(X) ≥ 0

Variance is always zero or positive, since it is the average of squared deviations.

7. Variance in Terms of PMF

Var(X) = Σ xi^2 * P(X=xi) - (Σ xi * P(X=xi))^2

First term: expected value of X²
Second term: square of the mean
Difference gives the spread around the mean

Numerical Example

Discrete random variable X with PMF:

x_i	1	2	3
P(X=x_i)	0.2	0.5	0.3

Step 1: Compute the mean

E[X] = 1*0.2 + 2*0.5 + 3*0.3 = 2.1

Step 2: Compute E[X²]

E[X^2] = 1^2*0.2 + 2^2*0.5 + 3^2*0.3 = 4.9

Step 3: Compute variance

Var(X) = E[X^2] - (E[X])^2 = 4.9 - (2.1)^2 = 0.49

Step 4: Interpretation

Variance = 0.49 → small spread around the mean
Standard deviation = √0.49 = 0.7 → average deviation from the mean

Joint PMF

a Joint PMF describes the probability distribution of two discrete random variable X and Y simultaneously.It hints you the what will be the probability that X takes the value xi and Y takes the value yi.

Formally, for discrete random variables X and Y:

pX,Y(xi,yj) = P(X=xi,Y=yj)

Properties of Joint PMF

Non-negativity:

p_X,Y(xi, yj) >= 0 for all xi, yj
Sum of all probabilities = 1:

Σi Σj pX,Y(xi, yj) = 1
Marginal PMF:

You can get the individual distributions of X or Y by summing over the other variable:

For X: pX(xi) = Σj p_X,Y(xi, yj)
For Y: pY(yj) = Σi p_X,Y(xi, yj)

Conditional Probability (under Joint PMF)

For two discrete random variables X and Y, the conditional probability of X = xi given Y = yj is:

P(X = xi | Y = yj) = P(X = xi AND Y = yj) / P(Y = yj)

Numerator: joint probability that both events happen: P(X = xi, Y = yj)
Denominator: probability of the condition: P(Y = yj)

It answers the question:

“If I know Y = yj has occurred, what is the probability that X = xi occurs?”

Independent Random Variable

In general given the random variable the PMF formula is given by,

P(X=x,Y=y,Z=z) = P(X=x)⋅P(Y=y∣X=x)⋅P(Z=z∣X=x,Y=y)

Random variables X, Y, Z are independent if,

P(X=x,Y=y,Z=z) = P(X=x) * P(Y=Y) * p(Z=z)

and so, PX|Y(x|y) = PX(x)

The Hat Problem

N people throw their hats in a box and then pick one at random X is the number of people who get their own hat.

-Find E[X]

xi = 1 if i selects own hat else 0

X = x1 _ x2 + x3 + .... + xn

p(xi = 1) = 1/n [each person will have probability of 1/n in picking own hat]

E[xi] = 1/n * 1 + (1 - 1/n) * 0 = 1/n
E[X] = n * 1/n = 1

so expected mean is 1 meaning that only one person in most cases will get their hat back!

let's calculate variance

Var(X) = E[X^2] - (E[X])^2

E[X]^2 = 1
E[X^2] = Σ xi^2 + Σxixj (where i!=j)

E[xi^2] = E[xi] = 1/n

E[X^2] = n * 1/n + 1/n * 1/(n-1) * (n^2 - n)

why 1/n * 1/(n-1) * n^2 - n ???

n^2 - n (since n^2 elements in total and we remove elements where i = j)
if P(X1X2 = 1) = P(X1 = 1)P(X2 = 1| X1 = 1) then,
it should be equal to 1/n * 1/(n - 1).

thus E[X^2] = 1 + 1 = 2

Var(X) = 2 - 1 = 1

thus having E[X] = 1 and Var(X) = 1 we can inference that there will be higher chance of k = 0, 1, or 2 being the number of person getting their own hat back.

Probability - Independence and Counting

Yadav Prasad G B — Wed, 01 Oct 2025 11:18:48 +0000

before getting into independence and counting let's see a very interesting topic.

Infinite Independent Events

consider a coin where the probability of getting a head is 1/2^n where n is the ith toss and each time a person does a coin flip the probability of getting heads will be 1/2, 1/4, 1/8.. and so on. what if we were told to find the probability of getting a first heads in a even flip number, we would have probably know the axioms but they don't comply with infinite events so what's the solution? you may ask so if given a problem where there is an infinite series of independent event and told to find the overall probability there comes the geometric series the formula is given in the picture below :

using this newly obtained formula we can derive the probability by finding the P({2, 4, 6,...}) = P(2) + P(4) + P(6) + ...
which is equals to 1/4 + 1/8 + 1/16.. = 1/2^2 + 1/2^4 + 1/2^6 + ...
which equals to 1/3 after applying the formula.

geometric series can be very useful for infinite independent event where they grow and can be ordered sequentially.

Independence

Definition: P(A | B) = P(A) occurrence of B doesn't provide any information about A's occurrence.

We Know that P (A n B) = P(B) * P(A|B)

with Independence : P(A n B) = P(B) * P(A)

with independence rule and multiplicative rule we can write as :

P(A n B | C) = P(A|C) * P(B|C)

where can we see these Independence ?? let's take an example of coin toss where p(getting a heads) = p. if you were told that the first 10 flips are heads does it change your belief about the probability of getting a heads in 11th toss? it's the same probability p for all the events so even though a person flips the toss 10 times that didn't change his initial beliefs about the coin flip in the 11th toss.

How does conditioning may affect independence

Two unfair coins A and B where P(getting an heads | A) = 0.9 and P(getting an heads | B) = 0.1, where the person can choose either coin with equal probability .

Once we know it is coin A, are tosses independent ? : Yes

If we do not know which coin it is, are tosses independent?? then what is the P(11th toss is Heads)? answer:= 1/2 * 0.9 + 1/2 * 0.1 = 1/2 (90% of getting heads from coin A and 10% of getting heads from coin B).

Now with conditional probability, P(11th toss is Heads | first 10 toss is heads) what is the probability of getting an heads in 11th toss?? answer : ≈ 0.9.

we can be certain that the coin A is being flipped from the get go but not 100% certain since coin B can also have a miracle chances of dropping this outcomes but it's just that it is very close to zero precisely (0.1)^10 so applying conditioning affects the relationship between two events it can make two independent events dependent or vice versa...

Basic Counting Principle

Permutations

Permutations : Number of ways of ordering n elements.

so understanding permutations, given that there are n elements in a set how many different arrangements can we made ?? : n! ways

now given a problem that a die is thrown 6 times what is the probability that 6 unique arrangement is made in 6 throws ie. {1,3,2,4,6,5} is one of the unique numbered arrangement

we can solve this by finding total arrangement resulting from throwing a die size times which is sample space (Ω) = 6 * 6 * 6 * 6 * 6 * 6 = 6^6 and from the sample space how many unique arrangements ca be made ??? from the above information we can say that n! ways arrangements can be made which is 6! so the resultant probability is,

|E|       6!
----   = ----    
|Ω|       6^6

Combinations

we know that we if taken n elements we can arrange it in n! ways so what about selecting k in n element set?

which can be rewritten as n!/(n-k)! ways.

assume that number of k - elements subsets if a given n-element set is denoted by C(n,k) or ⁿCₖ

now the number of k different arrangement out of chosen k elements subset from n-elements set is : k!

from the above equation :

k! × C(n,k) = n!/(n-k)!

C(n,k) = n!/[k!(n-k)!]

so there are two ways to come up with this formula n!/(n-k)!

Partitioning

if n elements is partitioned into m1, m2, m3 and m4 the total number of subsets made is

        n!
  ---------------
   m1! m2! m3! m4!

For example if a deck of 52 cards are partitioned into 4 groups of each 13 cards then the number of ways the cards can be distributed is

52!/(13! * 13! * 13! * 13!)

how does it work ??? let's understand intuitively

out of 52 cards if 13 can be selected (52 choose 13), after selecting the first 13 cards we have 39 cards remaining. We select another 13 cards (39 choose 13), after selecting the second group we have 26 cards remaining. We select another 13 cards (26 choose 13), the final 13 cards automatically form the last group (13 choose 13) = 1

So the total number of ways is:

(52 choose 13) × (39 choose 13) × (26 choose 13) × (13 choose 13)

= [52!/(13!×39!)] × [39!/(13!×26!)] × [26!/(13!×13!)] × [13!/(13!×0!)]

Now let's see the cancellation:

The 39! in the numerator of the second term cancels with the 39! in the denominator of the first term
The 26! in the numerator of the third term cancels with the 26! in the denominator of the second term
The 13! in the numerator of the fourth term cancels with the 13! in the denominator of the third term

After all the cancellations we get:

= 52! / (13! × 13! × 13! × 13!)

that's all for now folks, I will catch you up later.....

Probability - Introduction, axioms, Conditional and Bayes' Rule

Yadav Prasad G B — Mon, 29 Sep 2025 14:33:28 +0000

Probability

Probability is everywhere. It helps determine the likelihood of certain events given the whole sample space. By understanding probability, we can make better decisions based on the chances of favorable outcomes and reduce the impact of unfavorable events. Probability is widely used in areas like:

Climate prediction
Machine learning algorithms
Competitive programming
Medical research
Decision-making in everyday life

Terminology

Sample Space (Ω): List of all the elements in a set / all possible outcomes.
Mutually Exclusive Events: Two or more events that cannot occur at the same time in a single trial of an experiment.
Collectively Exhaustive Events: At least one of the events must occur in a single trial of an experiment.
Event: A set of favorable outcomes.

Understanding Probability Visually

Visualization helps intuitively understand probability. It works well for discrete sample spaces but can get complex for continuous probability. For example, consider two rolls of a tetrahedral die.

Another way to visualize:

Probability Axioms

Non-negativity: P(A) ≥ 0
Normalization: P(Ω) = 1
Additivity Rule: If A ∩ B = ∅, then P(A ∪ B) = P(A) + P(B)

Conditional Probability

Conditional probability calculates the likelihood of an event occurring given that another event has already occurred:

P(A | B) = P(A ∩ B) / P(B), where P(B) ≠ 0

Equivalently:

P(A ∩ B) = P(A | B) × P(B)

Let's understand how this formula can be derived intuitively with an example.

Example: Student & Exam

Event B: Student studies for the exam
Event A: Student passes the exam

Suppose:

70% of students study: P(B) = 0.7
Given they studied, 90% pass: P(A | B) = 0.9

Then:

P(A ∩ B) = 0.7 × 0.9 = 0.63

Meaning there’s a 63% chance that a randomly chosen student both studies and passes. 🎯

Let’s visualize this with a probability tree diagram:

Bayes’ Theorem

From conditional probability, we get Bayes’ theorem:

P(A | B) = [P(A) × P(B | A)] / P(B)

Where:

P(A ∩ B) = P(A) × P(B | A)

Terminology in Bayes’ Theorem:

Posterior (P(H | E)): Probability of the hypothesis given the evidence.
Prior (P(H)): Probability of the hypothesis before observing the evidence.
Likelihood (P(E | H)): Probability of the evidence given the hypothesis is true.
Marginal (P(E)): Probability of the evidence under all possible hypotheses.

Multiplicative Rule

For multiple events, the probability of all events occurring is:

P(A1 ∩ A2 ∩ ... ∩ An) = P(A1) × P(A2 | A1) × ... × P(An | A1 ∩ ... ∩ A(n-1))

This is a generalization of conditional probability for multiple events.

Now let us understand this visually using probability tree diagram given below,

that's all for now folks, I will catch you up later.....