Get href using css selector with Scrapy


I want to get the href value:

<span class="title">
  <a href=""></a>

I tried this:

Link = Link1.css('span[class=title] a::text').extract()[0]

But I just get the text inside the <a>. How can I get the link inside the href?

Asked By: Marco Dinatsoli



What you’re looking for is:

Link = Link1.css('span[class=title] a::attr(href)').extract()[0]

Since you’re matching a span “class” attribute also, you can even write

Link = Link1.css('span.title a::attr(href)').extract()[0]

Please note that ::text pseudo element and ::attr(attributename) functional pseudo element are NOT standard CSS3 selectors. They’re extensions to CSS selectors in Scrapy 0.20.

Edit (2017-07-20): starting from Scrapy 1.0, you can use .extract_first() instead of .extract()[0]

Link = Link1.css('span[class=title] a::attr(href)').extract_first()
Link = Link1.css('span.title a::attr(href)').extract_first()
Answered By: paul trmbrth
Link = Link1.css('span.title a::attr(href)').extract_first()

you can get more infomation from this

Answered By: Eddy

This will do the job:

Link = Link1.css('span.title a::attr(href)').extract()

Link will have the value :

Answered By: Jorgesys
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.