Constructing pandas DataFrame from values in variables gives "ValueError: If using all scalar values, you must pass an index"

Question:

This may be a simple question, but I can not figure out how to do this. Lets say that I have two variables as follows.

a = 2
b = 3

I want to construct a DataFrame from this:

df2 = pd.DataFrame({'A':a,'B':b})

This generates an error:

ValueError: If using all scalar values, you must pass an index

I tried this also:

df2 = (pd.DataFrame({'a':a,'b':b})).reset_index()

This gives the same error message.

Asked By: Nilani Algiriyage

||

Answers:

The error message says that if you’re passing scalar values, you have to pass an index. So you can either not use scalar values for the columns — e.g. use a list:

>>> df = pd.DataFrame({'A': [a], 'B': [b]})
>>> df
   A  B
0  2  3

or use scalar values and pass an index:

>>> df = pd.DataFrame({'A': a, 'B': b}, index=[0])
>>> df
   A  B
0  2  3
Answered By: DSM

You need to provide iterables as the values for the Pandas DataFrame columns:

df2 = pd.DataFrame({'A':[a],'B':[b]})
Answered By: ely

You can also use pd.DataFrame.from_records which is more convenient when you already have the dictionary in hand:

df = pd.DataFrame.from_records([{ 'A':a,'B':b }])

You can also set index, if you want, by:

df = pd.DataFrame.from_records([{ 'A':a,'B':b }], index='A')
Answered By: fAX

If you have a dictionary you can turn it into a pandas data frame with the following line of code:

pd.DataFrame({"key": d.keys(), "value": d.values()})
Answered By: ingrid

Maybe Series would provide all the functions you need:

pd.Series({'A':a,'B':b})

DataFrame can be thought of as a collection of Series hence you can :

  • Concatenate multiple Series into one data frame (as described here )

  • Add a Series variable into existing data frame ( example here )

Answered By: Rob

This is because a DataFrame has two intuitive dimensions – the columns and the rows.

You are only specifying the columns using the dictionary keys.

If you only want to specify one dimensional data, use a Series!

Answered By: danuker

You need to create a pandas series first. The second step is to convert the pandas series to pandas dataframe.

import pandas as pd
data = {'a': 1, 'b': 2}
pd.Series(data).to_frame()

You can even provide a column name.

pd.Series(data).to_frame('ColumnName')
Answered By: MLguy

If you intend to convert a dictionary of scalars, you have to include an index:

import pandas as pd

alphabets = {'A': 'a', 'B': 'b'}
index = [0]
alphabets_df = pd.DataFrame(alphabets, index=index)
print(alphabets_df)

Although index is not required for a dictionary of lists, the same idea can be expanded to a dictionary of lists:

planets = {'planet': ['earth', 'mars', 'jupiter'], 'length_of_day': ['1', '1.03', '0.414']}
index = [0, 1, 2]
planets_df = pd.DataFrame(planets, index=index)
print(planets_df)

Of course, for the dictionary of lists, you can build the dataframe without an index:

planets_df = pd.DataFrame(planets)
print(planets_df)
Answered By: k0L1081

You could try:

df2 = pd.DataFrame.from_dict({'a':a,'b':b}, orient = 'index')

From the documentation on the ‘orient’ argument: If the keys of the passed dict should be the columns of the resulting DataFrame, pass ‘columns’ (default). Otherwise if the keys should be rows, pass ‘index’.

Answered By: Matthew Connell

the input does not have to be a list of records – it can be a single dictionary as well:

pd.DataFrame.from_records({'a':1,'b':2}, index=[0])
   a  b
0  1  2

Which seems to be equivalent to:

pd.DataFrame({'a':1,'b':2}, index=[0])
   a  b
0  1  2
Answered By: S.V

I had the same problem with numpy arrays and the solution is to flatten them:

data = {
    'b': array1.flatten(),
    'a': array2.flatten(),
}

df = pd.DataFrame(data)
Answered By: MicheleDIncecco

Just pass the dict on a list:

a = 2
b = 3
df2 = pd.DataFrame([{'A':a,'B':b}])
Answered By: LeandroHumb

Pandas magic at work. All logic is out.

The error message "ValueError: If using all scalar values, you must pass an index" Says you must pass an index.

This does not necessarily mean passing an index makes pandas do what you want it to do

When you pass an index, pandas will treat your dictionary keys as column names and the values as what the column should contain for each of the values in the index.

a = 2
b = 3
df2 = pd.DataFrame({'A':a,'B':b}, index=[1])

    A   B
1   2   3

Passing a larger index:

df2 = pd.DataFrame({'A':a,'B':b}, index=[1, 2, 3, 4])

    A   B
1   2   3
2   2   3
3   2   3
4   2   3

An index is usually automatically generated by a dataframe when none is given. However, pandas does not know how many rows of 2 and 3 you want. You can however be more explicit about it

df2 = pd.DataFrame({'A':[a]*4,'B':[b]*4})
df2

    A   B
0   2   3
1   2   3
2   2   3
3   2   3

The default index is 0 based though.

I would recommend always passing a dictionary of lists to the dataframe constructor when creating dataframes. It’s easier to read for other developers. Pandas has a lot of caveats, don’t make other developers have to experts in all of them in order to read your code.

Answered By: firelynx

You may try wrapping your dictionary into a list:

my_dict = {'A':1,'B':2}
pd.DataFrame([my_dict])
   A  B
0  1  2
Answered By: NewBie

Convert Dictionary to Data Frame

col_dict_df = pd.Series(col_dict).to_frame('new_col').reset_index()

Give new name to Column

col_dict_df.columns = ['col1', 'col2']
Answered By: kamran kausar

You could try this:
df2 = pd.DataFrame.from_dict({‘a’:a,’b’:b}, orient = ‘index’)

Change your ‘a’ and ‘b’ values to a list, as follows:

a = [2]
b = [3]

then execute the same code as follows:

df2 = pd.DataFrame({'A':a,'B':b})
df2

and you’ll get:

    A   B
0   2   3
Answered By: Kalpana

I usually use the following to to quickly create a small table from dicts.

Let’s say you have a dict where the keys are filenames and the values their corresponding filesizes, you could use the following code to put it into a DataFrame (notice the .items() call on the dict):

files = {'A.txt':12, 'B.txt':34, 'C.txt':56, 'D.txt':78}
filesFrame = pd.DataFrame(files.items(), columns=['filename','size'])
print(filesFrame)

  filename  size
0    A.txt    12
1    B.txt    34
2    C.txt    56
3    D.txt    78
Answered By: Moritz Molch

simplest options ls :

dict  = {'A':a,'B':b}
df = pd.DataFrame(dict, index = np.arange(1) )
Answered By: user3505444

Another option is to convert the scalars into list on the fly using Dictionary Comprehension:

df = pd.DataFrame(data={k: [v] for k, v in mydict.items()})

The expression {…} creates a new dict whose values is a list of 1 element. such as :

In [20]: mydict
Out[20]: {'a': 1, 'b': 2}

In [21]: mydict2 = { k: [v] for k, v in mydict.items()}

In [22]: mydict2
Out[22]: {'a': [1], 'b': [2]}
Answered By: chenchuk

I tried transpose() and it worked.
Downside: You create a new object.

testdict1 = {'key1':'val1','key2':'val2','key3':'val3','key4':'val4'}

df = pd.DataFrame.from_dict(data=testdict1,orient='index')
print(df)
print(f'ID for DataFrame before Transpose: {id(df)}n')

df = df.transpose()
print(df)
print(f'ID for DataFrame after Transpose: {id(df)}')

Output

         0
key1  val1
key2  val2
key3  val3
key4  val4
ID for DataFrame before Transpose: 1932797100424

   key1  key2  key3  key4
0  val1  val2  val3  val4
ID for DataFrame after Transpose: 1932797125448

​```
Answered By: Hank Gordon
import pandas as pd
 a=2
 b=3
dict = {'A': a, 'B': b}

pd.DataFrame(pd.Series(dict)).T  
# *T :transforms the dataframe*

   Result:
    A   B
0   2   3
Answered By: M. John

To figure out the "ValueError" understand DataFrame and "scalar values" is needed.
To create a Dataframe from dict, at least one Array is needed.

IMO, array itself is indexed.
Therefore, if there is an array-like value there is no need to specify index.
e.g. The index of each element in [‘a’, ‘s’, ‘d’, ‘f’] are 0,1,2,3 separately.

df_array_like = pd.DataFrame({
    'col' : 10086,
    'col_2' : True,
    'col_3' : "'at least one array'",
    'col_4' : ['one array is arbitrary length', 'multi arrays should be the same length']}) 
print("df_array_like: n", df_array_like)

Output:

df_array_like: 
      col  col_2                 col_3                                   col_4
0  10086   True  'at least one array'           one array is arbitrary length
1  10086   True  'at least one array'  multi arrays should be the same length

As shows in the output, the index of the DataFrame is 0 and 1.
Coincidently same with the index of the array [‘one array is arbitrary length’, ‘multi arrays should be the same length’]

If comment out the ‘col_4’, it will raise

ValueError("If using all scalar values, you must pass an index")

Cause scalar value (integer, bool, and string) does not have index
Note that Index(…) must be called with a collection of some kind
Since index used to locate all the rows of DataFrame
index should be an array. e.g.

df_scalar_value = pd.DataFrame({
'col' : 10086,
'col_2' : True,
'col_3' : "'at least one array'"
}, index = ['fst_row','snd_row','third_row']) 
print("df_scalar_value: n", df_scalar_value)

Output:

df_scalar_value: 
              col  col_2                 col_3
fst_row    10086   True  'at least one array'
snd_row    10086   True  'at least one array'
third_row  10086   True  'at least one array'

I’m a beginner, I’m learning python and English.

Answered By: CN_Cabbage
Categories: questions Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.