How to calculate a logistic sigmoid function in Python?


This is a logistic sigmoid function:

enter image description here

I know x. How can I calculate F(x) in Python now?

Let’s say x = 0.458.

F(x) = ?

Asked By: Richard Knop



This should do it:

import math

def sigmoid(x):
  return 1 / (1 + math.exp(-x))

And now you can test it by calling:

>>> sigmoid(0.458)

Update: Note that the above was mainly intended as a straight one-to-one translation of the given expression into Python code. It is not tested or known to be a numerically sound implementation. If you know you need a very robust implementation, I’m sure there are others where people have actually given this problem some thought.

Answered By: unwind

another way

>>> def sigmoid(x):
...     return 1 /(1+(math.e**-x))
>>> sigmoid(0.458)
Answered By: ghostdog74

It is also available in scipy:

In [1]: from scipy.stats import logistic

In [2]: logistic.cdf(0.458)
Out[2]: 0.61253961344091512

which is only a costly wrapper (because it allows you to scale and translate the logistic function) of another scipy function:

In [3]: from scipy.special import expit

In [4]: expit(0.458)
Out[4]: 0.61253961344091512

If you are concerned about performances continue reading, otherwise just use expit.

Some benchmarking:

In [5]: def sigmoid(x):
  ....:     return 1 / (1 + math.exp(-x))

In [6]: %timeit -r 1 sigmoid(0.458)
1000000 loops, best of 1: 371 ns per loop

In [7]: %timeit -r 1 logistic.cdf(0.458)
10000 loops, best of 1: 72.2 µs per loop

In [8]: %timeit -r 1 expit(0.458)
100000 loops, best of 1: 2.98 µs per loop

As expected logistic.cdf is (much) slower than expit. expit is still slower than the python sigmoid function when called with a single value because it is a universal function written in C ( ) and thus has a call overhead. This overhead is bigger than the computation speedup of expit given by its compiled nature when called with a single value. But it becomes negligible when it comes to big arrays:

In [9]: import numpy as np

In [10]: x = np.random.random(1000000)

In [11]: def sigmoid_array(x):                                        
   ....:    return 1 / (1 + np.exp(-x))

(You’ll notice the tiny change from math.exp to np.exp (the first one does not support arrays, but is much faster if you have only one value to compute))

In [12]: %timeit -r 1 -n 100 sigmoid_array(x)
100 loops, best of 1: 34.3 ms per loop

In [13]: %timeit -r 1 -n 100 expit(x)
100 loops, best of 1: 31 ms per loop

But when you really need performance, a common practice is to have a precomputed table of the the sigmoid function that hold in RAM, and trade some precision and memory for some speed (for example: )

Also, note that expit implementation is numerically stable since version 0.14.0:

Answered By: Théo T

Here’s how you would implement the logistic sigmoid in a numerically stable way (as described here):

def sigmoid(x):
    "Numerically-stable sigmoid function."
    if x >= 0:
        z = exp(-x)
        return 1 / (1 + z)
        z = exp(x)
        return z / (1 + z)

Or perhaps this is more accurate:

import numpy as np

def sigmoid(x):  
    return np.exp(-np.logaddexp(0, -x))

Internally, it implements the same condition as above, but then uses log1p.

In general, the multinomial logistic sigmoid is:

def nat_to_exp(q):
    max_q = max(0.0, np.max(q))
    rebased_q = q - max_q
    return np.exp(rebased_q - np.logaddexp(-max_q, np.logaddexp.reduce(rebased_q)))

(However, logaddexp.reduce could be more accurate.)

Answered By: Neil G

Good answer from @unwind. It however can’t handle extreme negative number (throwing OverflowError).

My improvement:

def sigmoid(x):
        res = 1 / (1 + math.exp(-x))
    except OverflowError:
        res = 0.0
    return res
Answered By: czxttkl

Another way by transforming the tanh function:

sigmoid = lambda x: .5 * (math.tanh(.5 * x) + 1)
Answered By: dontloo

I feel many might be interested in free parameters to alter the shape of the sigmoid function. Second for many applications you want to use a mirrored sigmoid function. Third you might want to do a simple normalization for example the output values are between 0 and 1.


def normalized_sigmoid_fkt(a, b, x):
   Returns array of a horizontal mirrored normalized sigmoid function
   output between 0 and 1
   Function parameters a = center; b = width
   s= 1/(1+np.exp(b*(x-a)))
   return 1*(s-min(s))/(max(s)-min(s)) # normalize function to 0-1

And to draw and compare:

def draw_function_on_2x2_grid(x): 
    fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2)

    ax1.plot(x, normalized_sigmoid_fkt( .5, 18, x))

    ax2.plot(x, normalized_sigmoid_fkt(0.518, 10.549, x))

    ax3.plot(x, normalized_sigmoid_fkt( .7, 11, x))

    ax4.plot(x, normalized_sigmoid_fkt( .2, 14, x))
    plt.suptitle('Different normalized (sigmoid) function',size=10 )

    return fig


x = np.linspace(0,1,100)
Travel_function = draw_function_on_2x2_grid(x)

Sigmoid functions graph

Answered By: Philipp Schwarz

Tensorflow includes also a sigmoid function:

import tensorflow as tf

sess = tf.InteractiveSession()
x = 0.458
y = tf.sigmoid(x)

u = y.eval()
# 0.6125396

A numerically stable version of the logistic sigmoid function.

    def sigmoid(x):
        pos_mask = (x >= 0)
        neg_mask = (x < 0)
        z = np.zeros_like(x,dtype=float)
        z[pos_mask] = np.exp(-x[pos_mask])
        z[neg_mask] = np.exp(x[neg_mask])
        top = np.ones_like(x,dtype=float)
        top[neg_mask] = z[neg_mask]
        return top / (1 + z)
Answered By: Yash Khare

Use the numpy package to allow your sigmoid function to parse vectors.

In conformity with Deeplearning, I use the following code:

import numpy as np
def sigmoid(x):
    s = 1/(1+np.exp(-x))
    return s
Answered By: Diatche

A one liner…

In[1]: import numpy as np

In[2]: sigmoid=lambda x: 1 / (1 + np.exp(-x))

In[3]: sigmoid(3)
Out[3]: 0.9525741268224334
Answered By: cristiandatum
import numpy as np

def sigmoid(x):
    s = 1 / (1 + np.exp(-x))
    return s

result = sigmoid(0.467)

The above code is the logistic sigmoid function in python.
If I know that x = 0.467 ,
The sigmoid function, F(x) = 0.385. You can try to substitute any value of x you know in the above code, and you will get a different value of F(x).

Answered By: Dr Hazael Brown

Vectorized method when using pandas DataFrame/Series or numpy array:

The top answers are optimized methods for single point calculation, but when you want to apply these methods to a pandas series or numpy array, it requires apply, which is basically for loop in the background and will iterate over every row and apply the method. This is quite inefficient.

To speed up our code, we can make use of vectorization and numpy broadcasting:

x = np.arange(-5,5)
np.divide(1, 1+np.exp(-x))

0    0.006693
1    0.017986
2    0.047426
3    0.119203
4    0.268941
5    0.500000
6    0.731059
7    0.880797
8    0.952574
9    0.982014
dtype: float64

Or with a pandas Series:

x = pd.Series(np.arange(-5,5))
np.divide(1, 1+np.exp(-x))
Answered By: Erfan

you can calculate it as :

import math
def sigmoid(x):
  return 1 / (1 + math.exp(-x))

or conceptual, deeper and without any imports:

def sigmoid(x):
  return 1 / (1 + 2.718281828 ** -x)

or you can use numpy for matrices:

import numpy as np #make sure numpy is already installed
def sigmoid(x):
  return 1 / (1 + np.exp(-x))
Answered By: Ali

Below is the python function to do the same.

def sigmoid(x) :
    return 1.0/(1+np.exp(-x))
Answered By: Hardik Vagadia

You can simply declare 1 / np.exp(x) if putting before x confuse you.

def sigmoid(x):
     return 1 /(1 + 1 / np.exp(x))

Answered By: Rarblack
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.