Run Scrapy from a script
Question:
I’m trying to run my script without the command "scrapy crawl…", I’m following this documentation https://docs.scrapy.org/en/latest/topics/practices.html#run-scrapy-from-a-script, but my code is not working. Would appreciate the help!
import scrapy
from scrapy.crawler import CrawlerProcess
class misbeneficiosSpider(scrapy.Spider):
name = 'misbeneficios'
start_urls = ['https://productos.misbeneficios.com.uy/tv-y-audio',
'https://productos.misbeneficios.com.uy/tv-y-audio?p=2']
def parse(self, response):
for products in response.css('div.product-item-info'):
yield {
'name': products.css('a.product-item-link::text').get(),
'price': products.css('span.price::text').get().replace('U$Sxa0', '')#[:-3].upper()
}
next_page = response.css('a.action.next').attrib['href']
if next_page is not None:
yield response.follow(next_page, callback=self.parse)
process = CrawlerProcess(settings={
"FEEDS": {
"items.csv": {"format": "csv"},
},
})
process.crawl(misbeneficiosSpider)
process.start()
This is the error output I’m seeing:
2022-11-09 00:02:08 [scrapy.utils.log] INFO: Scrapy 2.7.1 started (bot: scrapybot)
2022-11-09 00:02:08 [scrapy.utils.log] INFO: Versions: lxml 4.9.1.0, libxml2 2.9.12, cssselect 1.2.0, parsel 1.7.0, w3lib 2.0.1, Twisted 22.10.0, Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:43:08) [MSC v.1926 32 bit (Intel)], pyOpenSSL 22.1.0 (OpenSSL 3.0.7 1 Nov 2022), cryptography 38.0.3, Platform Windows-10-10.0.22000-SP0
2022-11-09 00:02:08 [scrapy.crawler] INFO: Overridden settings:
{}
2022-11-09 00:02:08 [py.warnings] WARNING: C:UserscabreAppDataLocalProgramsPythonPython38-32libsite-packagesscrapyutilsrequest.py:231: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
return cls(crawler)
2022-11-09 00:02:08 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor
2022-11-09 00:02:08 [scrapy.extensions.telnet] INFO: Telnet Password: 034a39dcf0704cf3
2022-11-09 00:02:08 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats']
2022-11-09 00:02:09 [scrapy.core.downloader.handlers] ERROR: Loading "scrapy.core.downloader.handlers.http.HTTPDownloadHandler" for scheme "http"
Among others that I cant post
Answers:
It looks like with some minor error checking your code would work fine.
BTW there are two span.price tags per product card and I wasn’t sure which you wanted. So I think I just specified the first one.
For example:
import scrapy
from scrapy.crawler import CrawlerProcess
class misbeneficiosSpider(scrapy.Spider):
name = 'misbeneficios'
start_urls = ['https://productos.misbeneficios.com.uy/tv-y-audio']
def parse(self, response):
for products in response.css('div.product-item-info'):
price = products.css('span.price-wrapper span.price::text').get()
name = products.css('a.product-item-link::text').get()
if price and name:
yield {'name': name.strip(),
'price': price.strip('U$Sxa0').strip()}
next_page = response.css('a.next')
if next_page:
yield response.follow(next_page.attrib['href'], callback=self.parse)
process = CrawlerProcess(
settings={"FEEDS": {"items.csv": {"format": "csv"}}}
)
process.crawl(misbeneficiosSpider)
process.start()
OUTPUT : items.csv
name,price
"Smart TV LG 65"" UHD AI 65UP7750PSB","849,00"
"Smart TV Samsung 85"" Neo QLED UHD 4K","4.599,00"
"Smart TV LG 55"" OLED OLED55C2PSA","2.539,00"
"Smart TV LED Samsung 75"" UHD 4K UN75BU8000","1.999,00"
"Smart TV LG 32"" HD AI 32LM637BPSB","259,00"
"Smart TV TCL QLED 4K 55"" 55C825","1.599,00"
Smart TV Samsung Frame 55” UHD 4K SAQN55LS03BA,"1.679,00"
"Smart TV LED Smartlife 43"" SL-TV43FHDNXK","325,00"
"Smart TV Hometech 50"" UHD","339,00"
"Smart TV Hometech 55"" UHD","279,00"
"Smart TV Hometech 40"" FHD","275,00"
Barra de Sonido LG SP8A,"869,00"
"Smart TV LED 39"" HD Smartlife SL-TV39HDNX","265,00"
Smart TV LG OLED 48'' OLED48A1PSA,"1.359,00"
"Smart TV LG UHD 4K 60"" 60UQ8050PSB Al","989,00"
"Smart TV LG UHD 4K 65"" 65UQ8050PSB Al","1.275,00"
"Smart TV LG OLED 4K 65"" OLED65C2PSA AI","4.829,00"
"Smart TV LG 48"" OLED OLED48C2PSA","2.049,00"
"Smart TV LG 43"" NANOCELL 43NANO75SQA","549,00"
Proyector Samsung The Freestyle,"1.499,00"
"Smart TV Sony KD-75X80J 75""","1.999,00"
Parlante inalámbrico con Bluetooth Sony SRS-XB13,"59,00"
"Smart TV LG UHD 50"" 50UP7500PSF","539,00"
"Smart TV Philips 58"" 4K 58PUD6654/55 Borderless","899,00"
"Smart TV LG 43"" UHD 43UP7500PSF","465,00"
"Smart TV LG 86"" 4K 86UN8000PSB","3.990,00"
Parlante portátil Sony SRS-XB33,"159,00"
"Smart TV LG 75"" UHD AI 75UP7750PSB","2.095,00"
Equipo de audio de alta potencia V73D Sony MHC-V73D,"949,00"
Parlante profesional Xion Activo,"470,00"
Torre de sonido Sony MHC-V13,"359,00"
Barra de sonido Xion XI-BAR40,"55,00"
"Smart TV TCL 55"" UHD 55P615","579,00"
"Smart TV LG 65"" 4K OLED AI OLED65CXPSA","4.199,00"
"Smart TV TCL 50"" UHD 50P615","529,00"
"Smart TV Samsung 43"" FHD UN43T5300","359,00"
"Smart TV Philips 65"" 4K 65PUD6794/55 Ambilight","1.235,00"
"Smart TV LG OLED 77"" UHD AI OLED77G1","9.790,00"
"Smart TV Philips 32"" HD 32PHD6825/55","211,00"
"Smart TV LG 75"" 8K AI 75NANO95SNA","6.639,00"
"Smart TV Smartlife 50"" LED 4K","465,00"
"Smart TV Sony 55"" 4K KD-55X80J","999,00"
"Smart TV TCL 40"" FHD 40S65A","315,00"
Barra de sonido Sony HT-S100,"185,00"
Barra de Sonido LG SK1D,"189,00"
Parlante Sony SRS-XB23 Negro,"115,00"
Auriculares Apple Airpods 2 con estuche de carga,"166,00"
Auriculares Sony WH-CH510 Negro,"39,00"
Auriculares Sony WH-CH510 Blanco,"39,00"
Auriculares Sony WH-CH510 Azul,"39,00"
Torre de Sonido LG RN9 Xboom,"785,00"
"Smart TV LG 77"" 4K OLED AI OLED77GXPSA","9.190,00"
Torre de sonido LG RN7 Xboom,"559,00"
Torre de sonido LG RN5 Xboom,"345,00"
Parlante Samsung Giga Audio Party MX-T40,"325,00"
Parlante Energy Sistem Fabric Box 1+ grape,"27,00"
Parlante Energy Sistem Fabric Box 1+ blueberry,"25,00"
Parlante Sony MHC-V02,"279,00"
Parlante Billboard Bb730bt,"13,00"
Parlante Sony SRS-XB12 BT,"59,00"
Minicomponente LG Xboom CK99,"1.189,00"
Minicomponente LG Xboom CJ45,"299,00"
Minicomponente LG Xboom CM4360,"199,00"
Auricular SONY MDR-ZX110 Blanco,"22,00"
Auriculares Sony MDR-AS210 Blanco,"18,00"
Barra de sonido Xion XI-BAR70,"115,00"
Auriculares Sony Mdr-E9LP,"9,00"
Parlante Xion profesional XI-SD8BAT,"60,00"
Auricuares Sony MDR-ZX110 negro,"20,00"
Minicomponente LG XBOOM OL45,"315,00"
Auriculares Huawei Freebuds 4I Otter-Ct030,"93,00"
Minicomponente LG XBOOM CL88,"759,00"
Auriculares Sony MDR-EX15LP Rosado,"11,00"
Auriculares Sony MDR-EX15LP Azul,"11,00"
Auriculares Sony MDR-EX15LP negro,"11,00"
Auriculares Sony MDR-EX15LP Blanco,"11,00"
Auriculares Sony MDR-EX15LP violeta,"11,00"
"Smart TV Samsung 32"" UN32T4310 HD","265,00"
LOGS
2022-11-14 15:28:06 [scrapy.utils.log] INFO: Scrapy 2.7.1 started (bot: spiders)
2022-11-14 15:28:06 [scrapy.utils.log] INFO: Versions: lxml 4.9.1.0, libxml2 2.9.12, cssselect 1.2.0, parsel 1.7.0, w3lib 2.0.1, Twisted 22.10.0, Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53
:49) [MSC v.1932 64 bit (AMD64)], pyOpenSSL 22.1.0 (OpenSSL 3.0.7 1 Nov 2022), cryptography 38.0.3, Platform Windows-10-10.0.22621-SP0
2022-11-14 15:28:06 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'spiders',
'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
'SPIDER_MODULES': ['spiders.spiders'],
'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
'(KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36'}
2022-11-14 15:28:06 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor
2022-11-14 15:28:06 [scrapy.extensions.telnet] INFO: Telnet Password: b1466787c7ba254b
2022-11-14 15:28:06 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats']
2022-11-14 15:28:06 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2022-11-14 15:28:06 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2022-11-14 15:28:06 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2022-11-14 15:28:06 [scrapy.core.engine] INFO: Spider opened
2022-11-14 15:28:06 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2022-11-14 15:28:06 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2022-11-14 15:28:11 [filelock] DEBUG: Attempting to acquire lock 2089088199088 on C:Users4c83e38fb162f2e8738.tldextract.json.lock
2022-11-14 15:28:11 [filelock] DEBUG: Lock 2089088199088 acquired on C:Usersublicsuffix.org-tldsde84b5ca2167d4c83e38fb162f
2e8738.tldextract.json.lock
2022-11-14 15:28:11 [filelock] DEBUG: Attempting to release lock 2089088199088 on C:Usersagestldextract.suffix_cache/publicsuffix.org-tldsde84b5ca2167d
4c83e38fb162f2e8738.tldextract.json.lock
2022-11-14 15:28:11 [filelock] DEBUG: Lock 2089088199088 released on C:Usersvenvlibsite-packagestldextract.suffix_cache/publicsuffix.org-tldsde84b5ca2167d4c83e38fb162f
2e8738.tldextract.json.lock
2022-11-14 15:28:11 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://productos.misbeneficios.com.uy/tv-y-audio> (referer: None)
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Samsung 85" Neo QLED UHD 4K', 'price': '4.599,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 55" OLED OLED55C2PSA', 'price': '2.539,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LED Samsung 75" UHD 4K UN75BU8000', 'price': '1.999,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 32" HD AI 32LM637BPSB', 'price': '259,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV TCL QLED 4K 55" 55C825', 'price': '1.599,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Samsung Frame 55” UHD 4K SAQN55LS03BA', 'price': '1.679,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LED Smartlife 43" SL-TV43FHDNXK', 'price': '325,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Hometech 50" UHD', 'price': '339,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Hometech 40" FHD', 'price': '275,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 50" Nanocell 50NANO75SPA', 'price': '825,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Barra de Sonido LG SP8A', 'price': '869,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LED 39" HD Smartlife SL-TV39HDNX', 'price': '265,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': "Smart TV LG OLED 48'' OLED48A1PSA", 'price': '1.359,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG UHD 4K 60" 60UQ8050PSB Al', 'price': '989,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG UHD 4K 65" 65UQ8050PSB Al', 'price': '1.275,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG OLED 4K 65" OLED65C2PSA AI', 'price': '4.829,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 48" OLED OLED48C2PSA', 'price': '2.049,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 43" NANOCELL 43NANO75SQA', 'price': '549,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Proyector Samsung The Freestyle', 'price': '1.499,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Sony KD-75X80J 75"', 'price': '1.999,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante inalámbrico con Bluetooth Sony SRS-XB13', 'price': '59,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG UHD 50" 50UP7500PSF', 'price': '539,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Philips 58" 4K 58PUD6654/55 Borderless', 'price': '899,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG OLED 65" UHD AI OLED65A1', 'price': '2.495,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 43" UHD 43UP7500PSF', 'price': '465,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 86" 4K 86UN8000PSB', 'price': '3.990,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante portátil Sony SRS-XB33', 'price': '159,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Equipo de audio de alta potencia V73D Sony MHC-V73D', 'price': '949,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante profesional Xion Activo', 'price': '470,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Torre de sonido Sony MHC-V13', 'price': '359,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Barra de sonido Xion XI-BAR40', 'price': '55,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV TCL 55" UHD 55P615', 'price': '579,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 65" 4K OLED AI OLED65CXPSA', 'price': '4.199,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV TCL 50" UHD 50P615', 'price': '529,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Philips 65" 4K 65PUD6794/55 Ambilight', 'price': '1.235,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG OLED 77" UHD AI OLED77G1', 'price': '9.790,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Philips 32" HD 32PHD6825/55', 'price': '279,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 75" 8K AI 75NANO95SNA', 'price': '6.639,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 43" FHD AI 43LM6370PSB', 'price': '399,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Smartlife 50" LED 4K', 'price': '465,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Sony 55" 4K KD-55X80J', 'price': '999,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV TCL 40" FHD 40S65A', 'price': '315,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Barra de Sonido LG SK1D', 'price': '189,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Sony SRS-XB23 Negro', 'price': '115,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auriculares Apple Airpods 2 con estuche de carga', 'price': '166,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auriculares Sony WH-CH510 Negro', 'price': '39,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auriculares Sony WH-CH510 Blanco', 'price': '39,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auriculares Sony WH-CH510 Azul', 'price': '39,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Torre de Sonido LG RN9 Xboom', 'price': '785,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 77" 4K OLED AI OLED77GXPSA', 'price': '9.190,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Torre de sonido LG RN7 Xboom', 'price': '559,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Torre de sonido LG RN5 Xboom', 'price': '345,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Samsung Giga Audio Party MX-T40', 'price': '325,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Energy Sistem Fabric Box 1+ grape', 'price': '27,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Energy Sistem Fabric Box 1+ blueberry', 'price': '25,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Sony MHC-V02', 'price': '279,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Billboard Bb730bt', 'price': '13,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Sony SRS-XB12 BT', 'price': '59,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante OneBody LG Xboom OK75', 'price': '490,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Minicomponente LG Xboom CK99', 'price': '1.189,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Minicomponente LG Xboom CJ45', 'price': '299,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Minicomponente LG Xboom CM4360', 'price': '199,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auricular SONY MDR-ZX110 Blanco', 'price': '22,00'}
2022-11-14 15:28:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://productos.misbeneficios.com.uy/tv-y-audio?p=2> (referer: https://productos.misbeneficios.com.uy/tv-y-audio)
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auricuares Sony MDR-ZX310AP Blanco', 'price': '29,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-AS210 Blanco', 'price': '18,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Barra de sonido Xion XI-BAR70', 'price': '115,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony Mdr-E9LP', 'price': '9,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Parlante Xion profesional XI-SD8BAT', 'price': '60,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auricuares Sony MDR-ZX110 negro', 'price': '20,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Minicomponente LG XBOOM OL45', 'price': '315,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Huawei Freebuds 4I Otter-Ct030', 'price': '93,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Minicomponente LG XBOOM CL88', 'price': '759,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-EX15LP Rosado', 'price': '11,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-EX15LP negro', 'price': '11,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-EX15LP Blanco', 'price': '11,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-EX15LP violeta', 'price': '11,00'}
2022-11-14 15:28:15 [scrapy.core.engine] INFO: Closing spider (finished)
2022-11-14 15:28:15 [scrapy.extensions.feedexport] INFO: Stored json feed (76 items) in: a.json
2022-11-14 15:28:15 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 807,
'downloader/request_count': 2,
'downloader/request_method_count/GET': 2,
'downloader/response_bytes': 89887,
'downloader/response_count': 2,
'downloader/response_status_count/200': 2,
'elapsed_time_seconds': 8.887015,
'feedexport/success_count/FileFeedStorage': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2022, 11, 14, 23, 28, 15, 418905),
'httpcompression/response_bytes': 770913,
'httpcompression/response_count': 2,
'item_scraped_count': 76,
'log_count/DEBUG': 83,
'log_count/INFO': 11,
'request_depth_max': 1,
'response_received_count': 2,
'scheduler/dequeued': 2,
'scheduler/dequeued/memory': 2,
'scheduler/enqueued': 2,
'scheduler/enqueued/memory': 2,
'start_time': datetime.datetime(2022, 11, 14, 23, 28, 6, 531890)}
2022-11-14 15:28:15 [scrapy.core.engine] INFO: Spider closed (finished)
I’m trying to run my script without the command "scrapy crawl…", I’m following this documentation https://docs.scrapy.org/en/latest/topics/practices.html#run-scrapy-from-a-script, but my code is not working. Would appreciate the help!
import scrapy
from scrapy.crawler import CrawlerProcess
class misbeneficiosSpider(scrapy.Spider):
name = 'misbeneficios'
start_urls = ['https://productos.misbeneficios.com.uy/tv-y-audio',
'https://productos.misbeneficios.com.uy/tv-y-audio?p=2']
def parse(self, response):
for products in response.css('div.product-item-info'):
yield {
'name': products.css('a.product-item-link::text').get(),
'price': products.css('span.price::text').get().replace('U$Sxa0', '')#[:-3].upper()
}
next_page = response.css('a.action.next').attrib['href']
if next_page is not None:
yield response.follow(next_page, callback=self.parse)
process = CrawlerProcess(settings={
"FEEDS": {
"items.csv": {"format": "csv"},
},
})
process.crawl(misbeneficiosSpider)
process.start()
This is the error output I’m seeing:
2022-11-09 00:02:08 [scrapy.utils.log] INFO: Scrapy 2.7.1 started (bot: scrapybot)
2022-11-09 00:02:08 [scrapy.utils.log] INFO: Versions: lxml 4.9.1.0, libxml2 2.9.12, cssselect 1.2.0, parsel 1.7.0, w3lib 2.0.1, Twisted 22.10.0, Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:43:08) [MSC v.1926 32 bit (Intel)], pyOpenSSL 22.1.0 (OpenSSL 3.0.7 1 Nov 2022), cryptography 38.0.3, Platform Windows-10-10.0.22000-SP0
2022-11-09 00:02:08 [scrapy.crawler] INFO: Overridden settings:
{}
2022-11-09 00:02:08 [py.warnings] WARNING: C:UserscabreAppDataLocalProgramsPythonPython38-32libsite-packagesscrapyutilsrequest.py:231: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
return cls(crawler)
2022-11-09 00:02:08 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor
2022-11-09 00:02:08 [scrapy.extensions.telnet] INFO: Telnet Password: 034a39dcf0704cf3
2022-11-09 00:02:08 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats']
2022-11-09 00:02:09 [scrapy.core.downloader.handlers] ERROR: Loading "scrapy.core.downloader.handlers.http.HTTPDownloadHandler" for scheme "http"
Among others that I cant post
It looks like with some minor error checking your code would work fine.
BTW there are two span.price tags per product card and I wasn’t sure which you wanted. So I think I just specified the first one.
For example:
import scrapy
from scrapy.crawler import CrawlerProcess
class misbeneficiosSpider(scrapy.Spider):
name = 'misbeneficios'
start_urls = ['https://productos.misbeneficios.com.uy/tv-y-audio']
def parse(self, response):
for products in response.css('div.product-item-info'):
price = products.css('span.price-wrapper span.price::text').get()
name = products.css('a.product-item-link::text').get()
if price and name:
yield {'name': name.strip(),
'price': price.strip('U$Sxa0').strip()}
next_page = response.css('a.next')
if next_page:
yield response.follow(next_page.attrib['href'], callback=self.parse)
process = CrawlerProcess(
settings={"FEEDS": {"items.csv": {"format": "csv"}}}
)
process.crawl(misbeneficiosSpider)
process.start()
OUTPUT : items.csv
name,price
"Smart TV LG 65"" UHD AI 65UP7750PSB","849,00"
"Smart TV Samsung 85"" Neo QLED UHD 4K","4.599,00"
"Smart TV LG 55"" OLED OLED55C2PSA","2.539,00"
"Smart TV LED Samsung 75"" UHD 4K UN75BU8000","1.999,00"
"Smart TV LG 32"" HD AI 32LM637BPSB","259,00"
"Smart TV TCL QLED 4K 55"" 55C825","1.599,00"
Smart TV Samsung Frame 55” UHD 4K SAQN55LS03BA,"1.679,00"
"Smart TV LED Smartlife 43"" SL-TV43FHDNXK","325,00"
"Smart TV Hometech 50"" UHD","339,00"
"Smart TV Hometech 55"" UHD","279,00"
"Smart TV Hometech 40"" FHD","275,00"
Barra de Sonido LG SP8A,"869,00"
"Smart TV LED 39"" HD Smartlife SL-TV39HDNX","265,00"
Smart TV LG OLED 48'' OLED48A1PSA,"1.359,00"
"Smart TV LG UHD 4K 60"" 60UQ8050PSB Al","989,00"
"Smart TV LG UHD 4K 65"" 65UQ8050PSB Al","1.275,00"
"Smart TV LG OLED 4K 65"" OLED65C2PSA AI","4.829,00"
"Smart TV LG 48"" OLED OLED48C2PSA","2.049,00"
"Smart TV LG 43"" NANOCELL 43NANO75SQA","549,00"
Proyector Samsung The Freestyle,"1.499,00"
"Smart TV Sony KD-75X80J 75""","1.999,00"
Parlante inalámbrico con Bluetooth Sony SRS-XB13,"59,00"
"Smart TV LG UHD 50"" 50UP7500PSF","539,00"
"Smart TV Philips 58"" 4K 58PUD6654/55 Borderless","899,00"
"Smart TV LG 43"" UHD 43UP7500PSF","465,00"
"Smart TV LG 86"" 4K 86UN8000PSB","3.990,00"
Parlante portátil Sony SRS-XB33,"159,00"
"Smart TV LG 75"" UHD AI 75UP7750PSB","2.095,00"
Equipo de audio de alta potencia V73D Sony MHC-V73D,"949,00"
Parlante profesional Xion Activo,"470,00"
Torre de sonido Sony MHC-V13,"359,00"
Barra de sonido Xion XI-BAR40,"55,00"
"Smart TV TCL 55"" UHD 55P615","579,00"
"Smart TV LG 65"" 4K OLED AI OLED65CXPSA","4.199,00"
"Smart TV TCL 50"" UHD 50P615","529,00"
"Smart TV Samsung 43"" FHD UN43T5300","359,00"
"Smart TV Philips 65"" 4K 65PUD6794/55 Ambilight","1.235,00"
"Smart TV LG OLED 77"" UHD AI OLED77G1","9.790,00"
"Smart TV Philips 32"" HD 32PHD6825/55","211,00"
"Smart TV LG 75"" 8K AI 75NANO95SNA","6.639,00"
"Smart TV Smartlife 50"" LED 4K","465,00"
"Smart TV Sony 55"" 4K KD-55X80J","999,00"
"Smart TV TCL 40"" FHD 40S65A","315,00"
Barra de sonido Sony HT-S100,"185,00"
Barra de Sonido LG SK1D,"189,00"
Parlante Sony SRS-XB23 Negro,"115,00"
Auriculares Apple Airpods 2 con estuche de carga,"166,00"
Auriculares Sony WH-CH510 Negro,"39,00"
Auriculares Sony WH-CH510 Blanco,"39,00"
Auriculares Sony WH-CH510 Azul,"39,00"
Torre de Sonido LG RN9 Xboom,"785,00"
"Smart TV LG 77"" 4K OLED AI OLED77GXPSA","9.190,00"
Torre de sonido LG RN7 Xboom,"559,00"
Torre de sonido LG RN5 Xboom,"345,00"
Parlante Samsung Giga Audio Party MX-T40,"325,00"
Parlante Energy Sistem Fabric Box 1+ grape,"27,00"
Parlante Energy Sistem Fabric Box 1+ blueberry,"25,00"
Parlante Sony MHC-V02,"279,00"
Parlante Billboard Bb730bt,"13,00"
Parlante Sony SRS-XB12 BT,"59,00"
Minicomponente LG Xboom CK99,"1.189,00"
Minicomponente LG Xboom CJ45,"299,00"
Minicomponente LG Xboom CM4360,"199,00"
Auricular SONY MDR-ZX110 Blanco,"22,00"
Auriculares Sony MDR-AS210 Blanco,"18,00"
Barra de sonido Xion XI-BAR70,"115,00"
Auriculares Sony Mdr-E9LP,"9,00"
Parlante Xion profesional XI-SD8BAT,"60,00"
Auricuares Sony MDR-ZX110 negro,"20,00"
Minicomponente LG XBOOM OL45,"315,00"
Auriculares Huawei Freebuds 4I Otter-Ct030,"93,00"
Minicomponente LG XBOOM CL88,"759,00"
Auriculares Sony MDR-EX15LP Rosado,"11,00"
Auriculares Sony MDR-EX15LP Azul,"11,00"
Auriculares Sony MDR-EX15LP negro,"11,00"
Auriculares Sony MDR-EX15LP Blanco,"11,00"
Auriculares Sony MDR-EX15LP violeta,"11,00"
"Smart TV Samsung 32"" UN32T4310 HD","265,00"
LOGS
2022-11-14 15:28:06 [scrapy.utils.log] INFO: Scrapy 2.7.1 started (bot: spiders)
2022-11-14 15:28:06 [scrapy.utils.log] INFO: Versions: lxml 4.9.1.0, libxml2 2.9.12, cssselect 1.2.0, parsel 1.7.0, w3lib 2.0.1, Twisted 22.10.0, Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53
:49) [MSC v.1932 64 bit (AMD64)], pyOpenSSL 22.1.0 (OpenSSL 3.0.7 1 Nov 2022), cryptography 38.0.3, Platform Windows-10-10.0.22621-SP0
2022-11-14 15:28:06 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'spiders',
'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
'SPIDER_MODULES': ['spiders.spiders'],
'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
'(KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36'}
2022-11-14 15:28:06 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor
2022-11-14 15:28:06 [scrapy.extensions.telnet] INFO: Telnet Password: b1466787c7ba254b
2022-11-14 15:28:06 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats']
2022-11-14 15:28:06 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2022-11-14 15:28:06 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2022-11-14 15:28:06 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2022-11-14 15:28:06 [scrapy.core.engine] INFO: Spider opened
2022-11-14 15:28:06 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2022-11-14 15:28:06 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2022-11-14 15:28:11 [filelock] DEBUG: Attempting to acquire lock 2089088199088 on C:Users4c83e38fb162f2e8738.tldextract.json.lock
2022-11-14 15:28:11 [filelock] DEBUG: Lock 2089088199088 acquired on C:Usersublicsuffix.org-tldsde84b5ca2167d4c83e38fb162f
2e8738.tldextract.json.lock
2022-11-14 15:28:11 [filelock] DEBUG: Attempting to release lock 2089088199088 on C:Usersagestldextract.suffix_cache/publicsuffix.org-tldsde84b5ca2167d
4c83e38fb162f2e8738.tldextract.json.lock
2022-11-14 15:28:11 [filelock] DEBUG: Lock 2089088199088 released on C:Usersvenvlibsite-packagestldextract.suffix_cache/publicsuffix.org-tldsde84b5ca2167d4c83e38fb162f
2e8738.tldextract.json.lock
2022-11-14 15:28:11 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://productos.misbeneficios.com.uy/tv-y-audio> (referer: None)
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Samsung 85" Neo QLED UHD 4K', 'price': '4.599,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 55" OLED OLED55C2PSA', 'price': '2.539,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LED Samsung 75" UHD 4K UN75BU8000', 'price': '1.999,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 32" HD AI 32LM637BPSB', 'price': '259,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV TCL QLED 4K 55" 55C825', 'price': '1.599,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Samsung Frame 55” UHD 4K SAQN55LS03BA', 'price': '1.679,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LED Smartlife 43" SL-TV43FHDNXK', 'price': '325,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Hometech 50" UHD', 'price': '339,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Hometech 40" FHD', 'price': '275,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 50" Nanocell 50NANO75SPA', 'price': '825,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Barra de Sonido LG SP8A', 'price': '869,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LED 39" HD Smartlife SL-TV39HDNX', 'price': '265,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': "Smart TV LG OLED 48'' OLED48A1PSA", 'price': '1.359,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG UHD 4K 60" 60UQ8050PSB Al', 'price': '989,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG UHD 4K 65" 65UQ8050PSB Al', 'price': '1.275,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG OLED 4K 65" OLED65C2PSA AI', 'price': '4.829,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 48" OLED OLED48C2PSA', 'price': '2.049,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 43" NANOCELL 43NANO75SQA', 'price': '549,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Proyector Samsung The Freestyle', 'price': '1.499,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Sony KD-75X80J 75"', 'price': '1.999,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante inalámbrico con Bluetooth Sony SRS-XB13', 'price': '59,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG UHD 50" 50UP7500PSF', 'price': '539,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Philips 58" 4K 58PUD6654/55 Borderless', 'price': '899,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG OLED 65" UHD AI OLED65A1', 'price': '2.495,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 43" UHD 43UP7500PSF', 'price': '465,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 86" 4K 86UN8000PSB', 'price': '3.990,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante portátil Sony SRS-XB33', 'price': '159,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Equipo de audio de alta potencia V73D Sony MHC-V73D', 'price': '949,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante profesional Xion Activo', 'price': '470,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Torre de sonido Sony MHC-V13', 'price': '359,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Barra de sonido Xion XI-BAR40', 'price': '55,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV TCL 55" UHD 55P615', 'price': '579,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 65" 4K OLED AI OLED65CXPSA', 'price': '4.199,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV TCL 50" UHD 50P615', 'price': '529,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Philips 65" 4K 65PUD6794/55 Ambilight', 'price': '1.235,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG OLED 77" UHD AI OLED77G1', 'price': '9.790,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Philips 32" HD 32PHD6825/55', 'price': '279,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 75" 8K AI 75NANO95SNA', 'price': '6.639,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 43" FHD AI 43LM6370PSB', 'price': '399,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Smartlife 50" LED 4K', 'price': '465,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV Sony 55" 4K KD-55X80J', 'price': '999,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV TCL 40" FHD 40S65A', 'price': '315,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Barra de Sonido LG SK1D', 'price': '189,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Sony SRS-XB23 Negro', 'price': '115,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auriculares Apple Airpods 2 con estuche de carga', 'price': '166,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auriculares Sony WH-CH510 Negro', 'price': '39,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auriculares Sony WH-CH510 Blanco', 'price': '39,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auriculares Sony WH-CH510 Azul', 'price': '39,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Torre de Sonido LG RN9 Xboom', 'price': '785,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Smart TV LG 77" 4K OLED AI OLED77GXPSA', 'price': '9.190,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Torre de sonido LG RN7 Xboom', 'price': '559,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Torre de sonido LG RN5 Xboom', 'price': '345,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Samsung Giga Audio Party MX-T40', 'price': '325,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Energy Sistem Fabric Box 1+ grape', 'price': '27,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Energy Sistem Fabric Box 1+ blueberry', 'price': '25,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Sony MHC-V02', 'price': '279,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Billboard Bb730bt', 'price': '13,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante Sony SRS-XB12 BT', 'price': '59,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Parlante OneBody LG Xboom OK75', 'price': '490,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Minicomponente LG Xboom CK99', 'price': '1.189,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Minicomponente LG Xboom CJ45', 'price': '299,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Minicomponente LG Xboom CM4360', 'price': '199,00'}
2022-11-14 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio>
{'name': 'Auricular SONY MDR-ZX110 Blanco', 'price': '22,00'}
2022-11-14 15:28:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://productos.misbeneficios.com.uy/tv-y-audio?p=2> (referer: https://productos.misbeneficios.com.uy/tv-y-audio)
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auricuares Sony MDR-ZX310AP Blanco', 'price': '29,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-AS210 Blanco', 'price': '18,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Barra de sonido Xion XI-BAR70', 'price': '115,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony Mdr-E9LP', 'price': '9,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Parlante Xion profesional XI-SD8BAT', 'price': '60,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auricuares Sony MDR-ZX110 negro', 'price': '20,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Minicomponente LG XBOOM OL45', 'price': '315,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Huawei Freebuds 4I Otter-Ct030', 'price': '93,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Minicomponente LG XBOOM CL88', 'price': '759,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-EX15LP Rosado', 'price': '11,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-EX15LP negro', 'price': '11,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-EX15LP Blanco', 'price': '11,00'}
2022-11-14 15:28:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://productos.misbeneficios.com.uy/tv-y-audio?p=2>
{'name': 'Auriculares Sony MDR-EX15LP violeta', 'price': '11,00'}
2022-11-14 15:28:15 [scrapy.core.engine] INFO: Closing spider (finished)
2022-11-14 15:28:15 [scrapy.extensions.feedexport] INFO: Stored json feed (76 items) in: a.json
2022-11-14 15:28:15 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 807,
'downloader/request_count': 2,
'downloader/request_method_count/GET': 2,
'downloader/response_bytes': 89887,
'downloader/response_count': 2,
'downloader/response_status_count/200': 2,
'elapsed_time_seconds': 8.887015,
'feedexport/success_count/FileFeedStorage': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2022, 11, 14, 23, 28, 15, 418905),
'httpcompression/response_bytes': 770913,
'httpcompression/response_count': 2,
'item_scraped_count': 76,
'log_count/DEBUG': 83,
'log_count/INFO': 11,
'request_depth_max': 1,
'response_received_count': 2,
'scheduler/dequeued': 2,
'scheduler/dequeued/memory': 2,
'scheduler/enqueued': 2,
'scheduler/enqueued/memory': 2,
'start_time': datetime.datetime(2022, 11, 14, 23, 28, 6, 531890)}
2022-11-14 15:28:15 [scrapy.core.engine] INFO: Spider closed (finished)