Difficulties in extracting data from CSV and matching it with Product model

Question

I have a Django Python project where I’m trying to build an application where you can upload a csv file to then extract its values to generate a sales report pdf. Products can be added via the admin console and if those items appear (at the right place for now, eg item 4 in each row gets checked on) in the csv file they get extracted and added to the report (calculating the sum, storing the date of each purchase).

The csv file is as follows:

POS,Transaction id,Product,Quantity,Customer,Date
1,E100,TV,1,Test Customer,2022-09-19
2,E100,Laptop,3,Test Customer,2022-09-20
3,E200,TV,1,Test Customer,2022-09-21
4,E300,Smartphone,2,Test Customer,2022-09-22
5,E300,Laptop,5,New Customer,2022-09-23
6,E300,TV,1,New Customer,2022-09-23
7,E400,TV,2,ABC,2022-09-24
8,E500,Smartwatch,4,ABC,2022-09-25

I’m having 2 main problems, the first is that following a tutorial with someone using a Mac (I’m on Windows but also saving the csv file in Macintosh format didn’t fix this) the code he uses just doesn’t work for me. It literally returns an empty string:

with open(obj.csv_file.path, 'r') as f:
                reader = csv.reader(f)
                reader.__next__()
                for row in reader:
                    data = "".join(row)
                    data = data.split(';')
                    data.pop()

My workaround here is then to write the following code which generates a string separated by ‘;’:

for row in reader:
                    print(row,type(row))
                    data = " ".join(row)
                    data = data.split(";")

As part of this first problem I’m currently unable to grab the elements. I’m thinking that I probably need to convert the values into a list, but there’s a problem to that which is my main problem.

Going further in the code:

transaction_id = data[1]
                    product = data[2]
                    quantity = int(data[3])
                    customer = data[4]
                    date = parse_date(data[5])

                    print(transaction_id, product, quantity, customer, date)

                try:
                    product_obj = Product.objects.get(name__iexact=product)
                except Product.DoesNotExist:
                    product_obj = None
                
        return HttpResponse()

The terminal prints out:

Quit the server with CONTROL-C.
file is being uploaded
['1', 'E100', 'TV', '1', 'Test Customer', '9/19/2022'] <class 'list'>
['1 E100 TV 1 Test Customer 9/19/2022'] <class 'list'>
(…)
transaction_id = data[1]
IndexError: list index out of range

It turns out the product_obj always has the value None which it also had when I was playing around with the iteration where I occasionally could grad elements but never product_obj which is constantly set to None.

Asked By: a56z

||

Source

Answer 1

using csv.reader – no need to join and then split back row, etc.

with open(obj.csv_file.path, 'r') as f:
    rdr = csv.reader(f)
    next(rdr) # skip the header row

    # alternative 1
    for row in rdr:
        pos, transaction_id, product, quantity, customer, transaction_date = row
        # here work with product to check if exists

    # alternative 2
    for pos, transaction_id, product, quantity, customer, transaction_date in rdr:
        # here work with product to check if exists

Using csv.DictReader:

with open(obj.csv_file.path, 'r') as f:
    rdr = csv.DictReader(f)
    for row in rdr:
        product = row['Product'] # this will work even when column order/number has changed
        # here work with product to check if exists

Now I see this is your third question in a row on the same problem. You’ve got some good working code and some ill advise. Anyway, it doesn’t look like you actually try to understand the code you are given.

Answered By: buran

Difficulties in extracting data from CSV and matching it with Product model

Question:

Answers: