我无法执行以下操作:
from scrapy.selector import Selector
错误是:
来自scrapy.selector import Selector ImportError中的文件“ /Desktop/KSL/KSL/spiders/spider.py”,第1行,无法导入名称选择器
好像我的计算机上没有安装LXML,但是确实如此。另外,我认为这是scrapy内置的默认模块。也许不吧?
尝试改为导入HtmlXPathSelector。
from scrapy.selector import HtmlXPathSelector
然后使用.select()方法解析出你的html。例如,
sel = HtmlXPathSelector(response) site_names = sel.select('//ul/li')
示例将如下所示:
from scrapy.spider import BaseSpider from scrapy.selector import HtmlXPathSelector class DmozSpider(BaseSpider): name = "dmoz" allowed_domains = ["dmoz.org"] start_urls = [ "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/", "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/" ] def parse(self, response): sel = HtmlXPathSelector(response) sites = sel.select('//ul/li') for site in sites: title = site.select('a/text()').extract() link = site.select('a/@href').extract() desc = site.select('text()').extract() print title, link, desc
希望这可以帮助!