Convert DataFrame to Array of Objects

Question

Is there a way to convert a Pandas DataFrame into an array of objects (something that is more ‘comfortable’ to work with in JavaScript)?

I am using Facebook Prophet to run a timeseries forecast and return the data back to the client to do something with it.

I essentially want to take a DataFrame like this:

But, return something like this:

[
    {
        'ds': <value>,
        'trend': <value>,
        'yhat_lower': <value>,
        'yhat_upper': <value>
        ...
    },
    {
        'ds': <value>,
        'trend': <value>,
        'yhat_lower': <value>,
        'yhat_upper': <value>
        ...
    },
    ...
]

I have tried DataFrame.to_json() which is kind of close to what I need, but it presents other issues. I also tried DataFrame.to_dict() and that isn’t really what I want either. Same story fo DataFrame.to_records()

Do I really need to loop through the DataFrame manually to build up the list how I want it or is there some parameter/method I’m missing on getting a DataFrame to format as an array of objects with column names as the object key’s?

UPDATE

.to_dict() is close to what I want, but there’s still a nested object. Is there a way to get rid of that?

{'additive_terms': {0: 1821.6658106578184},
 'additive_terms_lower': {0: 1821.6658106578184},
 'additive_terms_upper': {0: 1821.6658106578184},
 'daily': {0: -904.5939055630084},
 'daily_lower': {0: -904.5939055630084},
 'daily_upper': {0: -904.5939055630084},
 'ds': {0: Timestamp('2016-01-01 00:00:00')},
 'multiplicative_terms': {0: 0.0},
 'multiplicative_terms_lower': {0: 0.0},
 'multiplicative_terms_upper': {0: 0.0},
 'trend': {0: 3959.7525337335633},
 'trend_lower': {0: 3959.7525337335633},
 'trend_upper': {0: 3959.7525337335633},
 'weekly': {0: 1382.1213748832024},
 'weekly_lower': {0: 1382.1213748832024},
 'weekly_upper': {0: 1382.1213748832024},
 'yearly': {0: 1344.1383413376243},
 'yearly_lower': {0: 1344.1383413376243},
 'yearly_upper': {0: 1344.1383413376243},
 'yhat': {0: 5781.418344391382},
 'yhat_lower': {0: -4262.772973874018},
 'yhat_upper': {0: 15333.709906373766}}

UPDATE 2

It looks like @busybear’s answer is what I want, however, I want it as an array of objects instead of a large object using the index as the key to the individual record:

{0: {'additive_terms': 1821.6658106578184,
  'additive_terms_lower': 1821.6658106578184,
  'additive_terms_upper': 1821.6658106578184,
  'daily': -904.5939055630084,
  'daily_lower': -904.5939055630084,
  'daily_upper': -904.5939055630084,
  'ds': Timestamp('2016-01-01 00:00:00'),
  'multiplicative_terms': 0.0,
  'multiplicative_terms_lower': 0.0,
  'multiplicative_terms_upper': 0.0,
  'trend': 3959.7525337335633,
  'trend_lower': 3959.7525337335633,
  'trend_upper': 3959.7525337335633,
  'weekly': 1382.1213748832024,
  'weekly_lower': 1382.1213748832024,
  'weekly_upper': 1382.1213748832024,
  'yearly': 1344.1383413376243,
  'yearly_lower': 1344.1383413376243,
  'yearly_upper': 1344.1383413376243,
  'yhat': 5781.418344391382,
  'yhat_lower': -4262.772973874018,
  'yhat_upper': 15333.709906373766},
 1: {'additive_terms': 1609.1847938356425,
  'additive_terms_lower': 1609.1847938356425,
  'additive_terms_upper': 1609.1847938356425,
  'daily': -904.5939055630084,
  'daily_lower': -904.5939055630084,
  'daily_upper': -904.5939055630084,
  'ds': Timestamp('2016-01-02 00:00:00'),
  'multiplicative_terms': 0.0,
  'multiplicative_terms_lower': 0.0,
  'multiplicative_terms_upper': 0.0,
  'trend': 3954.608221609561,
  'trend_lower': 3954.608221609561,
  'trend_upper': 3954.608221609561,
  'weekly': 1056.9172554279028,
  'weekly_lower': 1056.9172554279028,
  'weekly_upper': 1056.9172554279028,
  'yearly': 1456.8614439707483,
  'yearly_lower': 1456.8614439707483,
  'yearly_upper': 1456.8614439707483,
  'yhat': 5563.793015445203,
  'yhat_lower': -4892.457856774376,
  'yhat_upper': 15305.24188601227}}

Asked By: mwilson

||

Source

Answer 1

to_dict sounds like what you want actually. How is it not working for you? You could transpose your dataframe and take just the values of the dictionary. that will match what you expect:

data = df.T.to_dict()
list(data.values())

Answered By: busybear

Answer 2

Try This:

df.to_dict('records')

It gives you a list of dictionary without the index.

Answered By: anky

Answer 3

To do so:

df.to_json(orient="records")

That should get you to exactly what you need. Other orientations are split, index, and table.

For more information, see: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_json.html

Answered By: Rui Nian

Convert DataFrame to Array of Objects

Question:

Answers: