Python, Pandas : How to add formatting to excel file in case of MultiIndex?

Question:

I created a data frame with multiple indexes, (3 headers, and two indexes) I exported it to excel using the .to_excel() method, the first screenshot:
before

and I want to change the colors to look something like the second screenshot:
after

How can I achieve what I am looking for? I appreciate any help you can provide!
I found some help on other websites but not for the MultiIndex.
here is an example of my code:

import xlsxwriter
from xlsxwriter import Workbook
import pandas as pd
import numpy as np
from numpy import *
import ctypes
from openpyxl.workbook import Workbook
from docxtpl import DocxTemplate

laenge=['50','75','100','125','150','175','200','225','250','275','300']
TM=["S", "2TM", "3TM", "4TM", "5TM", "6TM", "8TM", "10TM", "12TM"]

spannung=['400V','200V']
MotorenList=["200HX","200UHX" ,"240HX", "310HX", "360UHX","564HX"]
DataToCalculat=['M_N','M_0','n_MAX','n_N','I_N','I_p','I_0']

columns=[np.array(spannung),np.array(MotorenList),np.array(DataToCalculat)]
columnsList=pd.MultiIndex.from_product(columns,names=['Voltage:','Motor:',''])

rows=[np.array(laenge),np.array(TM)]
indexList=pd.MultiIndex.from_product(rows,names=['h','TM'])

df=pd.DataFrame(np.random.randn(indexList.shape[0],columnsList.shape[0]),index=indexList, columns=columnsList)

writer= pd.ExcelWriter('test2.xlsx', engine='xlsxwriter')

for motor in MotorenList:
    
    if motor=='200HX' or motor=='200UHX':
        sheetName='RM 200'
        df_400V=pd.DataFrame(df[['400V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        df_200V=pd.DataFrame(df[['200V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        if motor=='200UHX':
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=df_400V.shape[1]+3)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=df_400V.shape[1]+3)
        else:
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=0)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=0)

    elif motor=='240HX' or motor=='240UHX':
        sheetName='RM 240'
        df_400V=pd.DataFrame(df[['400V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        df_200V=pd.DataFrame(df[['200V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        if motor=='240UHX':
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=df_400V.shape[1]+3)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=df_400V.shape[1]+3)
        else:
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=0)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=0)
    
    elif motor=='310HX' or motor=='310UHX':
        sheetName='RM 310'
        df_400V=pd.DataFrame(df[['400V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        df_200V=pd.DataFrame(df[['200V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        if motor=='310UHX':
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=df_400V.shape[1]+3)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=df_400V.shape[1]+3)
        else:
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=0)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=0)

    elif motor=='360HX' or motor=='360UHX':
        sheetName='RM 360'
        df_400V=pd.DataFrame(df[['400V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        df_200V=pd.DataFrame(df[['200V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        if motor=='360UHX':
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=df_400V.shape[1]+3)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=df_400V.shape[1]+3)
        else:
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=0)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=0)

    elif motor=='564HX' or motor=='564UHX':
        sheetName='RM 564'
        df_200V=pd.DataFrame(df[['200V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        df_200V=pd.DataFrame(df[['200V']].xs(motor,level='Motor:',axis=1,drop_level=False))
        if motor=='564UHX':
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=df_400V.shape[1]+3)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=df_400V.shape[1]+3)
        else:
            df_400V.to_excel(writer,sheet_name=sheetName,startrow=0,startcol=0)
            df_200V.to_excel(writer,sheet_name=sheetName,startrow=df_400V.shape[0]+5,startcol=0)

writer.close()
Asked By: MoMo

||

Answers:

Xlsxwriter cannot read existing data it only writes. Adding fill format after the to_excel writes would overwrite the existing cell values and you cannot read the values prior to rewrite back to the cel. Therefore using Openpyxl would be an option to add the fill without overwritting the existing values in the cell.
Rather than reloading the workbook/worksheet you can change the engine to Openpyxl and then incorporate the fill to your code.

Here is a code example;
The code example continues from your code imports, adding the PatternFill import at the bottom of the existing imports;

from openpyxl.styles import PatternFill

then adds a new function. Only two of the Motor sheets creations are show as they remain the same with the function added at the end as shown in the examples.
The ExcelWriter engine is also changed to ‘openpyxl’

writer = pd.ExcelWriter('test2.xlsx', engine='openpyxl')

The new function fill_colour is employed to set the background colours using Openpyxl which will fill all rows and columns in blue or gray per the ‘second screenshot’ using the Excel sheet.

fill_colour(writer.sheets[sheetName])

The function will look for and fill the whole sheet as if there are two Motors written to the Sheet. Fill only occurs if there is a value in the cell so there will be no empty cells filled for the first or second Motor if it does not exist. As the code is written, the fill function is called after each Motor write using the assigned sheetName, meaning for those sheets with two Motors the function is called twice and the cells with existing fills are refilled. This has minimal impact and it appears only RM 200 has two Motors. However the function can be moved so its called only after writting a Motor.

Filling is achieved by looping lists of cells for columns and iterating rows for the columns. Only one column, ‘B’ is iterated the other columns are filled at the same time using cell.offset.
The fill_colour function uses set row and column locations, e.g ‘C’ & ‘M’ as the merged Header cells and rows 4-103 and 108-207 as the filled rows. Most other cells are determined from these cells.
Three fill colours are used Blue, light Brown and Gray, these are set using PatternFill imported from Openpyxl

blueFill = PatternFill(start_color='FF8DB4E2', end_color='FF8DB4E2', fill_type='solid')
grayFill = PatternFill(start_color='FFD9D9D9', end_color='FFD9D9D9', fill_type='solid')
lghtBrwnFill = PatternFill(start_color='FFC4BD97', end_color='FFC4BD97', fill_type='solid')

The hex values can be changed if the colours are not correct.

...
from docxtpl import DocxTemplate
from openpyxl.styles import PatternFill  # Included import


### Add fill_colour function
def fill_colour(sheet):
    ### Blue Fill
    col_list = ['C', 'M']
    row_list = [1, 105]
    for l in col_list:
        for j in row_list:
            fill_cell = sheet[f'{l}{j}']
            if fill_cell.value is not None:
                sheet[f'{l}{j}'].fill = blueFill

    ### Gray & Brown Fill
    ### Fill header cells
    head_dict = {4: 103, 108: 207}
    for hrow, v in head_dict.items():
        ### Create list of cells in the Header section to fill
        ### Columns C to S for row 3 or 107 (excludes cols J, K & L)
        head_list = [f'{chr(i)}{hrow - 1}' for i in range(ord('C'), ord('S') + 1)
                     if chr(i) != 'J' and chr(i) != 'K' and chr(i) != 'L']
        ### Background fill each cell in the head_list
        for hcell in head_list:
            gray_cell = sheet[hcell]
            if gray_cell.value is not None:
                gray_cell.fill = grayFill

        ### Add the cells for the 'Motor' row in light  brown
        motor_list = [f'C{hrow-2}', f'M{hrow-2}']
        for mot_cell in motor_list:
            brn_cell = sheet[mot_cell]
            if brn_cell.value is not None:
                brn_cell.fill = lghtBrwnFill

        ### Fill cells in columns A, B and K, L if used
        for row in sheet.iter_rows(min_col=2, max_col=2, min_row=hrow, max_row=v):
            for cell in row:
                if cell.value is not None:
                    cell.fill = grayFill
                col_list = [-1, 9, 10]
                for col in col_list:
                    if cell.offset(column=col).value is not None:
                        cell.offset(column=col).fill = grayFill

blueFill = PatternFill(start_color='FF8DB4E2', end_color='FF8DB4E2', fill_type='solid')
grayFill = PatternFill(start_color='FFD9D9D9', end_color='FFD9D9D9', fill_type='solid')
lghtBrwnFill = PatternFill(start_color='FFC4BD97', end_color='FFC4BD97', fill_type='solid')

laenge = ['50', '75', '100', '125', '150', '175', '200', '225', '250', '275', '300']
TM = ["S", "2TM", "3TM", "4TM", "5TM", "6TM", "8TM", "10TM", "12TM"]

spannung = ['400V', '200V']
MotorenList = ["200HX", "200UHX", "240HX", "310HX", "360UHX", "564HX"]
DataToCalculat = ['M_N', 'M_0', 'n_MAX', 'n_N', 'I_N', 'I_p', 'I_0']

columns = [np.array(spannung), np.array(MotorenList), np.array(DataToCalculat)]
columnsList = pd.MultiIndex.from_product(columns, names=['Voltage:', 'Motor:', ''])

rows = [np.array(laenge), np.array(TM)]
indexList = pd.MultiIndex.from_product(rows, names=['h', 'TM'])

df = pd.DataFrame(np.random.randn(indexList.shape[0], columnsList.shape[0]), index=indexList, columns=columnsList)

### Change engine to 'openpyxl'
writer = pd.ExcelWriter('test2.xlsx', engine='openpyxl')

for motor in MotorenList:

    if motor == '200HX' or motor == '200UHX':
        sheetName = 'RM 200'
        df_400V = pd.DataFrame(df[['400V']].xs(motor, level='Motor:', axis=1, drop_level=False))
        df_200V = pd.DataFrame(df[['200V']].xs(motor, level='Motor:', axis=1, drop_level=False))
        if motor == '200UHX':
            df_400V.to_excel(writer, sheet_name=sheetName, startrow=0, startcol=df_400V.shape[1] + 3)
            df_200V.to_excel(writer, sheet_name=sheetName, startrow=df_400V.shape[0] + 5, startcol=df_400V.shape[1] + 3)
        else:
            df_400V.to_excel(writer, sheet_name=sheetName, startrow=0, startcol=0)
            df_200V.to_excel(writer, sheet_name=sheetName, startrow=df_400V.shape[0] + 5, startcol=0)

        ### Add the fill_colour function, with worksheet created from sheetName
        fill_colour(writer.sheets[sheetName])

    elif motor == '240HX' or motor == '240UHX':
        sheetName = 'RM 240'
        df_400V = pd.DataFrame(df[['400V']].xs(motor, level='Motor:', axis=1, drop_level=False))
        df_200V = pd.DataFrame(df[['200V']].xs(motor, level='Motor:', axis=1, drop_level=False))
        if motor == '240UHX':
            df_400V.to_excel(writer, sheet_name=sheetName, startrow=0, startcol=df_400V.shape[1] + 3)
            df_200V.to_excel(writer, sheet_name=sheetName, startrow=df_400V.shape[0] + 5, startcol=df_400V.shape[1] + 3)
        else:
            df_400V.to_excel(writer, sheet_name=sheetName, startrow=0, startcol=0)
            df_200V.to_excel(writer, sheet_name=sheetName, startrow=df_400V.shape[0] + 5, startcol=0)

        fill_colour(writer.sheets[sheetName])

    elif motor == '310HX' or motor == '310UHX':
        sheetName = 'RM 310'    
...

###The rest of the code is the same as your sample code with the fill_colour function for each Motor as shown in the two above Motors.

Example of the updated Sheet, this link shows the top of the first Sheet
‘RM 200’ sheet

Answered By: moken
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.