What is the fastest way to generate 100 000 000 normally distributed values?

Question:

I am struggling with generating a large list having normal values mean=5.357 and std-dev=2.37

Original list

org_list=[3.65, 4.11, 1.63, 6.7, 9, 7.61, 5.5, 2.9, 3.99, 8.48]

Candidates methods

Currently I am trying to use the following modules: random.normalvariate, random.gauss and np.normal

Tryings and goal

First I tried them on a relatively reduced length:
For random.normalvariate I got:

new_list_normalvariate = [random.normalvariate(5.357, 2.37) for x in range(1000)]
print(new_list_normalvariate[0:10])
print('mean = ', np.mean(new_list_normalvariate))
print('std = ', np.std(new_list_normalvariate))

>>> [6.576049386450241, 8.62262371117091, 4.921246966899101, 6.751587914411607, 5.6042223736139105, 4.493753810671122, 7.868066836581562, 6.299169672752275, 6.081202725113191, 7.27255885543875]
>>> mean =  5.3337034248054875
>>> std =  2.4124820216611336

For random.gauss I got:

new_list_gauss = [random.gauss(5.357, 2.37) for x in range(1000)]
print(new_list_gauss[0:10])
print('mean = ', np.mean(new_list_gauss))
print('std = ', np.std(new_list_gauss))

>>> [4.160280814524453, 8.376767324676795, 8.476968737124544, 6.050223384914485, 2.6635671201126785, 2.4441297408189167, 7.624650437282289, 7.5957096799039485, 1.990806588702878, 1.7821756994741982]
>>> mean =  5.347638951117946
>>> std =  2.374617608342891

And for np.normal I got:

new_list_np_normal = [np.random.normal(5.357, 2.37) for x in range(1000)]
print(new_list_np_normal[0:10])
print('mean = ', np.mean(new_list_np_normal))
print('std = ', np.std(new_list_np_normal))

>>> [4.294445875786478, 4.930900785615266, 8.244969311017886, 3.380908919026986, 3.636133194752361, 6.191836517294145, 5.17400630491519, 3.16529157634111, 1.9176117359394778, 8.269659173531764]
>>> mean =  5.417575775284877
>>> std =  2.373787525312793

Problem

When I make the size very large (e.g. 10000000), it seems that each of above methods take long time.

new_list_gauss = [random.gauss(5.357, 2.37) for x in range(10000000)]

So I want a best method allowing me to generate a large number of normal values with a low time cost.

Asked By: Khaled DELLAL

||

Answers:

If you use the size argument of np.normal, it is pretty fast. I.e. np.random.normal(5.357, 2.37, size=(10000000)) executes in less than half a second on my machine, compared to 24 seconds for the list comprehension approach

Answered By: Alex P
Categories: questions Tags:
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.