我正在尝试获取下载链接并下载文件。
我有一个包含以下链接的日志文件:
http://www.downloadcrew.com/article/18631-aida64 http://www.downloadcrew.com/article/4475-sumo http://www.downloadcrew.com/article/2174-iolo_system_mechanic_professional ... ...
我有这样的代码:
import urllib, time from bs4 import BeautifulSoup from selenium import webdriver from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By f = open("dcrewtest.txt") for line in f.readlines(): try: driver.find_element_by_xpath("//div/div[2]/div[2]/div[2]/div[3]/div/a/img").click() time.sleep(8) except: pass url = line.encode pageurl = urllib.urlopen(url).read() soup = BeautifulSoup(pageurl) for a in soup.select("h1#articleTitle"): print a.contents[0].strip() for b in soup.findAll("th"): if b.text == "Date Updated:": print b.parent.td.text elif b.text == "Developer:": print c.parent.td.text
到目前为止,我不知道如何获取下载链接并下载它。可以使用selenium下载文件吗?
根据文档,您应该配置FirefoxProfile为自动下载具有指定内容类型的文件。这是在txt文件中使用第一个URL的示例,该文件将exe文件保存在当前目录中:
FirefoxProfile
exe
import os from selenium import webdriver fp = webdriver.FirefoxProfile() fp.set_preference("browser.download.folderList",2) fp.set_preference("browser.download.manager.showWhenStarting",False) fp.set_preference("browser.download.dir", os.getcwd()) fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/x-msdos-program") driver = webdriver.Firefox(firefox_profile=fp) driver.get("http://www.downloadcrew.com/article/18631-aida64") driver.find_element_by_xpath("//div[@class='downloadLink']/a/img").click()
注意,我也简化了xpath。