Joint Probability in Statistics and Probability Theory

When we wish to perform the statistical measurement of two events that are likely to occur at the same time, there we use the joint probability. 

Assume that there is an event “X” and another event “Y” if they both are occurring at the same time, we say that the joint probability is the probability of Y occurring at the time of the event X.

Assume that you throw a dice and you obtain the following set of numbers:

The first event X   =  {1, 2, 3, 4, 5, 6}

However, another event occurs in which you obtain the set of odd numbers:

Y  =  {1, 3, 5}

Now, here the two events are occurring, we get the joint distribution of two random variables: {1, 3, 5}. This is how we get the joint probability distribution. We will understand the joint probability formula relying on the joint distribution. 

Moving ahead with the joint distribution function, we will go through the marginal probability looking at the marginal distribution probability.

Joint Probability Formula

The notation of a joint probability takes various forms, however, the following joint probability formula talks about the following intersection of probability events:

P   =  XY

(Image will be uploaded soon)         

So, we notice that two events, i.e., X and Y are the two varying events that intersect. Also, P (X and Y), P(X, Y) is the joint probability of events X and Y.

Here, we discussed the joint distribution of random variables X and Y. It’s because these variables are defined on a probability space, the joint probability distribution for X and Y is a joint probability distribution that gives the probability of events that are common under the given set of values in these two variables, respectively. 

Joint Distribution Between Two Random Variables

Probability is a domain that closely relates to statistics and it deals with the likelihood of the phenomenon or an event occurring. For calculating the probability of an event, we quantify the probability as a number between 0 and 1, and inclusive, where 0 pertains to an impossible event of occurrence and 1 represents the certainty of an event.

For instance, the probability of picking a black card from a deck of cards is ½ = 0.5. This means that there is an equal opportunity of obtaining a black and drawing a red; since a deck comprises 52 cards, out of which 26 are red and 26 are black, therefore, we have a 50-50 probability of drawing a black card vs a red card.

Therefore, we understand that a joint probability is a statistical measure of two events (black or a red card) occurring at the same time, and can only be applied to conditions where more than one event can occur simultaneously. 

Now, for instance, from a deck of 52 cards, the joint probability between two random variables of drawing a card that is both black and 8 is P (8 ∩ black) = 2/52 = 1/26 because a deck of cards has two black eights - the eight of clubs and another eight of spades. 

Since both the events "8" and "black" are independent events in this example, you can also apply another formula to calculate the joint probability, which is as follows:

P (8 ∩ black) = P (8) x P (black)  =  4/52 x 26/52  =  1/26

Joint Probability Distribution

We can represent the joint probability distribution either in terms of a joint cumulative distribution function or a joint probability density function (when talking about the continuous variables) or joint probability mass function (while considering discrete variables). 

Also, we apply the joint distribution to find the two types of probability distributions, viz: the marginal probability distribution that gives the probabilities for any one of the variables without making any reference to a specific range of values for the other variables.

However, another is the conditional probability distribution that gives the probabilities for any subset of the variables conditional on specific values of the remaining variables.

Joint Probability Distribution Example

Let us suppose that two urns contain twice as many black balls as red balls and no other than these, let’s say, one ball is randomly selected from each urn, while the other two draws are independent of each other. 

So, we consider two events as A and B, which are discrete random variables linked with the outcomes of the ball drawn from the first urn and second urn, respectively. 

Now, the probability of drawing a black ball from either of the urns is 2/3, and that the probability of drawing a red ball is 1/3. The joint probability table for the joint probability distribution is presented in the following manner:

Joint Probability Table

Here, we notice that each of the four inner cells in the above joining probability table shows the probability of a specific combination of results from the two draws; therefore, these probabilities are the joint distribution. 

However, talking about any individual cell, the probability of a particular combination occurring for independent picks is the multiple of the probability of the specified result for A and B. The probabilities obtained in these four cells always add to 1, as it is always sure for probability distributions.

Further, observing the final row and the final column, we get the marginal distribution probability for A, along with the marginal distribution probability for B. 

For instance, for an event, A, the first of these cells provides the sum of the probabilities for A being black, regardless of whichever possibility for B in the column above the cell occurs as ⅔.

Therefore, the marginal probability distribution for A returns A's probabilities that is unconditional on B, is a marginal probability of the table.

Marginal Probability Formula

Consider an example for determining the marginal probability:

Now, we will calculate the marginal distribution of pet preference among males and females:

Solution:

Step 1: Count the total number of people (male or female). In this case, the total is given in the right-hand column (11 individuals).

Step 2: Now, count the number of people who like any type of pet and then turn the ratio into a probability:

People who prefer a pet cat: 5/11 

People who prefer dog as a pet: 11/17

People who prefer monkeys: 12/23

We conclude that a joint probability is a theoretical probability that refers to the probability of two events occurring at the same time. In simple words, the joint probability is the likelihood of two events occurring together.