How do I make a correlation matrix for each subset of a column of my pandas dataframe? Question: Here’s the head of my dataframe: There are 100 different loggers and 10 different years. I want to subset the table by logger and find the Pearson correlation values for year by avg_max_temp, avg_min_temp, and tot_precipitation for …
How to slice and calculate the pearson correlation coefficient between one big and small array with "overlapping" windows arrays Question: Suppose I have two very simple arrays with numpy: import numpy as np reference=np.array([0,1,2,3,0,0,0,7,8,9,10]) probe=np.zeros(3) I would like to find which slice of array reference has the highest pearson’s correlation coefficient with array probe. To …
Compute correlations of several vectors Question: I have several pairs of vectors (arranged as two matrices) and I want to compute the vector of their pairwise correlation coefficients (or, better yet, angles between them – but since correlation coefficient is its cosine, I am using numpy.corrcoef): np.array([np.corrcoef(m1[:,i],m2[:,i])[0,1] for i in range(m1.shape[1])]) I wonder if there …
np.corrcoef returns only nan Question: i have an array norm_array=np.array([[1, 133, 1, 5.73, 5.09, 11.12, 10.16, 3.38, 15, 3, 8, 7, 4, 5, 1, 6, 1, 2, 18, 12, 48], [1, 185, 0, 4.34, 3.66, 18.23, 14.91, 0, 21, 15, 11, 2, 4, 4, 5, 4, 9, 8, 27, 14, 47], [1, 133, 1, 5.92, …
correlation matrix with group-by and sort Question: I am trying calculate correlation matrix with groupby and sort. I have 100 companies from 11 industries. I would like to group by industry and sort by their total assets (atq), and then calculate the correlation of data.pr_multi with this order. however, when I do sort and groupby, …
How to convert every dictionary in a list to a nested dictionary in python? Question: I am using pandas and numpy libraries, to calculate the pearson correlation of two simple lists. The output of the below code is the matrix of correlation: import numpy as np import pandas as pd x = np.array([0, 1, 2, …
Homemade pearson's correlation implementation returning 0.999…2 when passing two of the same sets of data to it Question: I was getting fed up of scipy and numpy, and decided to go ahead and work on another implementation, based on a SO answer somewhere. from statistics import pstdev, mean def pearson(x, y): sx = [] sy …
