Why does CSV file contain a blank line in between each data line when outputting with Dictwriter in Python
Question:
I am using DictWriter to output data in a dictionary to a csv file. Why does the CSV file have a blank line in between each data line? It’s not a huge deal, but my dataset is big and doesn’t fit into one csv file because it has too many lines since the “double-spacing” doubles the number of lines in the file.
My code for writing to the dictionary is:
headers=['id', 'year', 'activity', 'lineitem', 'datum']
output = csv.DictWriter(open('file3.csv','w'), delimiter=',', fieldnames=headers)
output.writerow(dict((fn,fn) for fn in headers))
for row in rows:
output.writerow(row)
Answers:
I just tested your snippet, and their is no double spacing line here. The end-of-line are rn, so what i would check in your case is:
- your editor is reading correctly DOS file
- no n exist in values of your rows dict.
(Note that even by putting a value with n, DictWriter automaticly quote the value.)
By default, the classes in the csv module use Windows-style line terminators (rn) rather than Unix-style (n). Could this be what’s causing the apparent double line breaks?
If so, in python 2 you can override it in the DictWriter constructor:
output = csv.DictWriter(open('file3.csv','w'), delimiter=',', lineterminator='n', fieldnames=headers)
From csv writer documentation:
If csvfile is a file object, it should be opened with newline=''
In other words, when opening the file you pass newline='' as a parameter.
You can also use a with statement to close the file when you’re done writing to it.
Tested example below:
from __future__ import with_statement # not necessary in newer versions
import csv
headers=['id', 'year', 'activity', 'lineitem', 'datum']
with open('file3.csv','w', newline='') as fou:
output = csv.DictWriter(fou,delimiter=',',fieldnames=headers)
output.writerow(dict((fn,fn) for fn in headers))
output.writerows(rows)
Changing the ‘w’ (write) in this line:
output = csv.DictWriter(open('file3.csv','w'), delimiter=',', fieldnames=headers)
To ‘wb’ (write binary) fixed this problem for me:
output = csv.DictWriter(open('file3.csv','wb'), delimiter=',', fieldnames=headers)
Credit to @dandrejvv for the solution in the comment on the original post above.
I am using DictWriter to output data in a dictionary to a csv file. Why does the CSV file have a blank line in between each data line? It’s not a huge deal, but my dataset is big and doesn’t fit into one csv file because it has too many lines since the “double-spacing” doubles the number of lines in the file.
My code for writing to the dictionary is:
headers=['id', 'year', 'activity', 'lineitem', 'datum']
output = csv.DictWriter(open('file3.csv','w'), delimiter=',', fieldnames=headers)
output.writerow(dict((fn,fn) for fn in headers))
for row in rows:
output.writerow(row)
I just tested your snippet, and their is no double spacing line here. The end-of-line are rn, so what i would check in your case is:
- your editor is reading correctly DOS file
- no n exist in values of your rows dict.
(Note that even by putting a value with n, DictWriter automaticly quote the value.)
By default, the classes in the csv module use Windows-style line terminators (rn) rather than Unix-style (n). Could this be what’s causing the apparent double line breaks?
If so, in python 2 you can override it in the DictWriter constructor:
output = csv.DictWriter(open('file3.csv','w'), delimiter=',', lineterminator='n', fieldnames=headers)
From csv writer documentation:
If csvfile is a file object, it should be opened with
newline=''
In other words, when opening the file you pass newline='' as a parameter.
You can also use a with statement to close the file when you’re done writing to it.
Tested example below:
from __future__ import with_statement # not necessary in newer versions
import csv
headers=['id', 'year', 'activity', 'lineitem', 'datum']
with open('file3.csv','w', newline='') as fou:
output = csv.DictWriter(fou,delimiter=',',fieldnames=headers)
output.writerow(dict((fn,fn) for fn in headers))
output.writerows(rows)
Changing the ‘w’ (write) in this line:
output = csv.DictWriter(open('file3.csv','w'), delimiter=',', fieldnames=headers)
To ‘wb’ (write binary) fixed this problem for me:
output = csv.DictWriter(open('file3.csv','wb'), delimiter=',', fieldnames=headers)
Credit to @dandrejvv for the solution in the comment on the original post above.