How to export product information to csv and create 6 columns?

Question

I created a python program using selenium and I want to export the information to csv and create 6 columns(Product Title, Product Price, Product weight, Product dimension, Product ASIN, Items in cart).

The first 5 product information are from the first Amazon page and the 6th information are from the cart page. Now I print the first 5 columns and export the 6th column to csv but I want to create a csv with all these 6 columns with these product information.

from selenium import webdriver
from time import sleep
from selenium.webdriver.common.keys import Keys
import csv
proxies = {
    'http': 'http://31.173.209.111:8080',
    'https': 'http://118.179.60.110:8080',
}

url = "https://www.amazon.com/Haggar-Hidden-Comfort-Waist-Plain/dp/B0018Q3BRO/ref=sr_1_3?ie=UTF8&qid=1518006207&sr=8-3&keywords=trousers+for+men"

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % proxies)

driver = webdriver.Chrome(executable_path="C:\UsersAndreiDownloadschromedriver_win32chromedriver.exe",
                          chrome_options=chrome_options)
driver.get(url)

product_title = driver.find_elements_by_xpath('//*[@id="productTitle"][1]')
product_price = driver.find_elements_by_xpath('//*[@id="priceblock_ourprice"][1]')
product_weight = driver.find_elements_by_xpath('//*[@id="detailBullets_feature_div"]/ul/li[2]/span/span')
product_dimension = driver.find_elements_by_xpath('//*[@id="detailBullets_feature_div"]/ul/li[1]/span/span')
product_asin = driver.find_elements_by_xpath('//*[@id="detailBullets_feature_div"]/ul/li[3]/span/span')
product_reviews = driver.find_elements_by_xpath('//*[@id="detailBullets_averageCustomerReviews"]/span/span')
prod_title = [x.text for x in product_title]
prod_price = [x.text for x in product_price]
prod_weight = [x.text for x in product_weight]
prod_dimension = [x.text for x in product_dimension]
prod_asin = [x.text for x in product_asin]
prod_reviews = [x.text for x in product_reviews]
print(prod_title[0])
print(prod_price[0])
print(prod_weight[1])
print(prod_dimension[1])
print(prod_asin[1])

try:
    sizemenu = driver.find_element_by_id('dropdown_selected_size_name')
    sizemenu.click()
    select = driver.find_element_by_id('size_name_1')  # medium size
    select.click()
except:
    print('no select')

sleep(3)

driver.find_element_by_xpath('//*[@id="submit.add-to-cart"]/span/input').click()

# driver.find_element_by_xpath('//*[@id="smartShelfAddToCartContinue"]/span/input')
sleep(3)
try:
    driver.execute_script("document.getElementById('smartShelfAddToCartNative').click()")
except:
    print('no continue button')

try:
    driver.find_element_by_xpath('//*[@id="hlb-view-cart"]/span/a').click()
except:
    print('no cart button')

sleep(3)

# items_cart = driver.find_element_by_xpath('//div[@class="a-alert-content"]/span')

driver.find_element_by_xpath('//*[@id="a-autoid-0-announce"]/span[2]').click()
driver.find_element_by_xpath('//*[@id="dropdown1_9"]').click()
quantity_xpath = '//*[@id="activeCartViewForm"]/div[2]/div/div[4]/div/div[3]/div/div/input'
quantity_el = driver.find_element_by_xpath(quantity_xpath)
quantity_el.send_keys("999" + Keys.ENTER)

sleep(2)

items_cart = driver.find_elements_by_xpath('//div[@class="a-alert-content"]/span')
items_in_cart = [x.text for x in items_cart]

csvfile='products_final.csv'
# Assuming res is a flat list
with open(csvfile, "w") as output:
    writer = csv.writer(output, lineterminator='n')
    writer.writerow(items_in_cart[1])

Asked By: ryy77

||

Source

Answer 1

If you are only outputting one header row and one row of features, it is probably easiest to just create these each as a list. For example:

header = ['Product Title', 'Product Price', 'Product weight', 'Product dimension', 'Product ASIN', 'Items in cart']

and, following the indexes you’ve described,

data = [prod_title[0], prod_price[0], prod_weight[1], prod_dimension[1], prod_asin[1], items_in_cart[1]]

you can than write these results like this:

with open(csvfile, "w") as output:
    writer = csv.writer(output)
    writer.writerow(header)
    writer.writerow(data)

Answered By: Andrew Brown

How to export product information to csv and create 6 columns?

Question:

Answers: