List of pandas DataFrames: After pickle deserialization empty

Question

I built a simple container class which holds a list of pd.DataFrames.
I fill the list by a setter method and save this container class using pickle.
Saving and loading works well if done right serial in one script.

PROBLEM: Deserialization returns the list of dataframes as empty.

A minimal example: executing the code beneath returns the dataframes of l. Then, commenting out the ‘instantiate/save’ section beneath only rerunning the ‘loading’, will return an empty list.

import pandas as pd
import pickle

data1 = [[12.0, 13.1], [14.0, 15.0]]
data2 = [[24.0, 25.1], [26.0, 27.0]]

df1 = pd.DataFrame(data1, columns=["T11", "T12"])
df2 = pd.DataFrame(data2, columns=["T11", "T12"])

class container():

    l = []

    def append_l(self, df):
        self.l.append(df)

# instantiate/save
device = container()
device.append_l(df1)
device.append_l(df2)

with open("save.pkl", "wb") as outfile:
    pickle.dump(device, outfile)

# loading
with open("save.pkl", "rb") as infile:
    loaded = pickle.load(infile)

print(loaded.l)

I expect that pickle only saves references no objects. References are running out of scope after one session and getting deleted. Therefore the list references are pointing to empty spaces.

How could I ressolve this issue?

Asked By: Wendel

||

Source

Answer 1

Redefine the list as an instance attribute instead of a class one:

class container():

    def __init__():
        self.l = []

    def append_l(self, df):
        self.l.append(df)

Answered By: SiP

List of pandas DataFrames: After pickle deserialization empty

Question:

Answers: