Random selection with conditional probabilities

Question

I have list say y = [1, 2, 3, 4, 6, 7, 8, 9, 5, 23, 12, 24, 43, 10] and I want to make a random selection from it with conditional probability. A number greater than 10 in the list has a probability of say 0.8 of being selected while the rest have probability 0.2 of being selected.

Asked By: EArwa

||

Source

Answer 1

try this :
it uses np.random.choice from a list a member of elements with a list that content the propabilité of each element the for loop will generate that list from the input list:

import numpy as np
y = [1, 2, 3, 4, 6, 7, 8, 9, 5, 23, 12, 24, 43, 10]
t=[]
sup=0
inf=0
for j in y:
    if j>10:
        sup=sup+1
    else:
        inf=inf+1
print(sup)
print(inf)
p=[]
for i in y:
    if i>10:
        p.append(1/sup)
    else:
        p.append(0/inf)
print(p)
x=np.random.choice(y, 100, p=p)

print(x)

output

[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.25, 0.25, 0.25, 0.25, 0.0]
[12 23 23 43 23 12 12 43 24 12]

another output with probability 0.5 and 0.5 :

[0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.125, 0.125, 0.125, 0.125, 0.05]
[24 24 43  8  4 12 23 24  6  5]

another one with :

[0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.2, 0.2, 0.2, 0.2, 0.02]
[24 43 23 24  7 24 12 12 43 24]

Answered By: Ghassen

Answer 2

Since random.choice provides a uniform distribution, you will have to work in two steps. First select between the groups of values (below 10 and above 10). Then select a value within the group.

To get different probabilities between groups, you can create a list with the appropriate number of repetitions of each group. For example, for 0.2 and 0.8 you would have 2 instance of the “below10” group and 8 instances of the “above10” group in the list. This will transform the regular distribution into a weighted distribution relative to each group.

import random

treshold = 10
y       = [1, 2, 3, 4, 6, 7, 8, 9, 5, 23, 12, 24, 43, 10]
group1  = [v for v in y if v <  treshold]
group2  = [v for v in y if v >= treshold]

def getValue():
    group = random.choice([group1]*2 + [group2]*8)
    return random.choice(group)

To test if the distribution is as required, you can use the function a large number of times and calculate how many times a value in each group was selected.

lowCount  = 0
highCount = 0
N         = 10000
for _ in range(N):
    v = getValue()
    if v < treshold:
        lowCount += 1
    else:
        highCount += 1
print(round(lowCount/N,2),round(highCount/N,2))

# 0.2 0.8

If you only ever have 2 groups, you could use a simple if-else statement for the selection:

def getValue():
    return random.choice(group1) if random.random() <= 0.2 else random.choice(group2)

EDIT For a single value (lets say 23) with a probability of 0.9, the approach is the same:

y       = [1, 2, 3, 4, 6, 7, 8, 9, 5, 23, 12, 24, 43, 10]
group1  = [23]
group2  = [v for v in y if v not in group1]
def getValue():
    return random.choice(group1) if random.random() <= 0.9 else random.choice(group2)

lowCount  = 0
highCount = 0
N         = 10000
for _ in range(N):
    v = getValue()
    if v == 23:        # <== same condition as the grouping rule.
        lowCount += 1
    else:
        highCount += 1
print(round(lowCount/N,2),round(highCount/N,2))

# 0.9 0.1

You have to adjust your testing loop accordingly however

Answered By: Alain T.

Random selection with conditional probabilities

Question:

Answers: