Looking for good indexing and sparse matrix methods to create a matrix from existing one Question: I have a large numpy array A of shape M*3, whose elements of each row are unique, non-negative integers ranging from 0 to N – 1. In fact, each row corresponds to a triangle in my finite element analysis. …
Fill sparse row of a dataframe with the existing values in the column Question: I have a dataframe of the following types: CurrentDf = pd.DataFrame(np.array([[5, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0], [7, 0, 0, 0, 1, 0, 2, 0, 0, 0, 0], [8, 0, 0, 0, 1, 0, 3, 0, 0, …
How to load a sparse matrix in Python from an MMF file Question: How can I load data of a sparse matrix form (MMF) file, and then carry out a linear algebra operation on that matrix? The MMF file is in the following form: 1 1 8.530175905456780E+008 7 1 1.257919566068602E+008 12 1 3.841933299769016E+002 13 1 …
Pandas : ValueError ( any way to convert Sparse[float64, 0.0] dtypes to float64 datatype ) Question: I have a dataframe X_train to which i am concatenating a couple of another dataframe. This second & third dataframe is obtained from sparse matrix which has been been generated by a TF-IDF VEctorizer q1_train_df = pd.DataFrame.sparse.from_spmatrix(q1_tdidf_train,index=X_train.index,columns=q1_features) q2_train_df = …
How to delete small elements in sparse matrix in Python's SciPy? Question: I have a question that is quite similiar to Sean Laws example that you can find here: https://seanlaw.github.io/2019/02/27/set-values-in-sparse-matrix/ In my case, I want to delete all the elements in a sparse csr matrix, which have an absolute value smaller than some epsilon. First …
scipy csr_matrix: understand indptr Question: Every once in a while, I get to manipulate a csr_matrix but I always forget how the parameters indices and indptr work together to build a sparse matrix. I am looking for a clear and intuitive explanation on how the indptr interacts with both the data and indices parameters when …
R internal handling of sparse matrices Question: I have been comparing the performance of several PCA implementations from both Python and R, and noticed an interesting behavior: While it seems impossible to compute the PCA of a sparse matrix in Python (the only approach would be scikit-learn’s TruncatedSVD, yet it does not support the mean-centering …
How to Find Indices where multiple vectors all are zero Question: Beginner pySpark question here: How do I find the indices where all vectors are zero? After a series of transformations, I have a spark df with ~2.5M rows and a tfidf Sparse Vector of length ~262K. I would like to perform PCA dimensionality reduction …
How to find zero elements in a sparse matrix Question: I know that scipy.sparse.find(A) returns 3 arrays I,J,V each of them containing the rows, columns, and values of the nonzero elements respectively. What i want is a way to do the same (except the V array) for all zero elements without having to iterate through …
How can I give row and column names to Scipy's csr_matrix? Question: I don’t know if it’s possible, and it’s possibly a naive question, but how can I set the equivalent of R’s rownames() and colnames() to a scipy.sparse.csr.csr_matrix ? I saw that my_matrix.dtype.names doesn’t work here, and I can’t find any “index” equivalend for …
