How to implement CSV Writer in a named temporary – Python?

Question

Basically, I need final a csv file, whose contents look like Header, and then the data:
(Boto3 upload_file does the job of writing temporary file, into csv)

Expectation:

Name,Code
Adam,12

I am able to get this, by using:

with tempfile.NamedTemporaryFile(mode="a+t", suffix =".csv", delete=True) as fileName:
   for data in allData:
      fileName.write(data)
      fileName.flush()

But, when I am using csv.writer:

writer = csv.write(fileName)
with tempfile.NamedTemporaryFile(mode="a+t", suffix =".csv", delete=True) as fileName:
   for data in allData:
      writer.writerow(data)

Reality:

N,a,m,e","C,o,d,e
A,d,a,m","1,2"

Using Python 3.8

allData looks like: [{'name': 'Adam', 'code': '12', 'points': 9.7, 'age': '34'}, {{'name': 'Sam', 'code': '13', 'points': 8.4, 'age': '34'}]. (There are 1000 data like this, and using a context filters out the values only.)

A data looks like: Adam,12

Need help writing this with csv writer. I have seen references of StringIO being used to fix similar cases, but do not know how to implement. Is there a way out?

Asked By: layman

||

Source

Answer 1

This doesn’t have anything to do with the temporary directory.

Here is part of the introduction about CSV writer objects (bold added):

A row must be an iterable of strings or numbers for Writer objects and a dictionary mapping fieldnames to strings or numbers… for DictWriter objects

But you are passing in a single string: "Adam,12". So what’s going on?

Strings themselves are iterables over the characters in the string:

>>> for character in "Adam":
...     print(character)
...
A
d
a
m

And since Python doesn’t have a distinct character class, each of those single-character values is also a string. So strings are iterables of strings.

When you give an iterable of strings to csvwriter.writerow(), it writes each string in the iterable to its own column. In this case, that means one-character strings taken from the input.

To get the behaviour you’re after, you’ll need to pass something like ["Adam", 12] (not "Adam,12") to the CSV writer. I’m not sure where your inputs are coming from, so there might be a better way to do this, but you could split your strings on the comma as you pass them in:

for data in allData:
    writer.writerow(data.split(","))

This works because "Adam,12".split(",") returns ["Adam", "12"]. Note that this won’t work properly if your inputs have multiple commas.

Edit:

If allData is a list of dictionaries, as you show in the edited question, you might be better off with a DictWriter instead of a regular writer:

Create an object which operates like a regular writer but maps dictionaries onto output rows. The fieldnames parameter is a sequence of keys that identify the order in which values in the dictionary passed to the writerow() method are written to file f.

In other words, something like this:

with open("foo.csv", "w") as f:
    writer = csv.DictWriter(f, ["name", "code", "points", "age"])
    for data in [{"name": "Adam", "code": 12, "points": 0, "age": 18}]:
        writer.writerow(data)

Answered By: Chris

How to implement CSV Writer in a named temporary – Python?

Question:

Answers: