当我尝试从该网站访问数据时,我尝试使用Selenium自动化从网站中获取数据
run: Starting ChromeDriver (v2.9.248315) on port 15621 Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/http/entity/ContentType at org.openqa.selenium.remote.HttpCommandExecutor$EntityWithEncoding.<init>(HttpCommandExecutor.java:411) at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:306) at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:66) at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:568) at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:240) at org.openqa.selenium.chrome.ChromeDriver.startSession(ChromeDriver.java:181) at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:126) at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:139) at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:160) at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:149) at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:106) at pocmandi.PocMandi.main(PocMandi.java:32) Caused by: java.lang.ClassNotFoundException: org.apache.http.entity.ContentType at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 12 more
这是我的鳕鱼
package pocmandi; import com.gargoylesoftware.htmlunit.BrowserVersion; import java.util.StringTokenizer; import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.firefox.FirefoxDriver; import org.openqa.selenium.htmlunit.HtmlUnitDriver; import org.openqa.selenium.support.ui.Select; import java.sql.*; import org.openqa.selenium.chrome.ChromeDriver; public class PocMandi { Statement st = null; Connection cn = null; public static void main(String args[]) throws InterruptedException, ClassNotFoundException, SQLException { int j = 0; String htmlTableText = null; System.setProperty("webdriver.chrome.driver", "C:\\Users\\SHAKTI\\Desktop\\JarFiles\\chromedriver_win32\\chromedriver.exe"); WebDriver driver = new ChromeDriver(); String commodity = "Jo"; String commo[] = {"Paddy", "Rice", "Jwar", "Barley", "Corn", "Wheat", "Jo", "Bejhar", "Jai", "Urad", "Moong", "Chana", "Matar"}; for (String com : commo) { String sDate = "27/03/2014"; String url = "http://www.upmandiparishad.in/commodityWiseAll.aspx"; driver.get(url); Thread.sleep(5000); new Select(driver.findElement(By.id("ctl00_ContentPlaceHolder1_ddl_commodity"))).selectByVisibleText(com); driver.findElement(By.id("ctl00_ContentPlaceHolder1_txt_rate")).sendKeys(sDate); Thread.sleep(3000); driver.findElement(By.id("ctl00_ContentPlaceHolder1_btn_show")).click(); Thread.sleep(5000); WebElement findElement = driver.findElement(By.id("ctl00_ContentPlaceHolder1_GridView1")); // WebElement find=driver.findElement(By.id("ctl00_ContentPlaceHolder1_ddl_commodity")); htmlTableText = findElement.getText(); // String html=find.getText(); // do whatever you want now, This is raw table values. htmlTableText = htmlTableText.replace("S.No.DistrictMarketPrice", ""); htmlTableText = htmlTableText.replaceAll("\\s(\\d+\\s[A-Z])", "\n$1"); htmlTableText = htmlTableText.replaceAll("(?=(.*?[ ]){4,}).*?[\n\r]", ""); htmlTableText = htmlTableText.replace("S.No. District Market Price", ""); System.out.println(htmlTableText); String s[] = htmlTableText.split(""); StringTokenizer str = new StringTokenizer(htmlTableText); while (str.hasMoreTokens()) // for(int i=0;i<s.length;i++) // if(str.hasMoreElements()) { String no = str.nextElement().toString(); String city = str.nextElement().toString(); String mandi = str.nextElement().toString(); String price = str.nextElement().toString(); Class.forName("com.mysql.jdbc.Driver"); Connection cn = DriverManager.getConnection("jdbc:mysql://localhost:3306/mandi", "root", ""); //insert them into the database PreparedStatement ps = cn.prepareStatement("insert into commoditydemo values(?,?,?,?,?,?)"); ps.setString(1, no); ps.setString(2, city); ps.setString(3, mandi); ps.setString(4, price); ps.setString(5, com); ps.setString(6, "0"); j = ps.executeUpdate(); cn.close(); } } driver.close(); driver.quit(); if (j == 1) { System.out.println("data inserted"); } else { System.out.println("not inserted"); } } }
如何获取我的输出并删除此异常?
提前致谢
为了检查您的代码,我尝试了以下操作:
@Test public void test1() throws Exception { System.setProperty("webdriver.chrome.driver", "t:\\Others\\chromedriver.exe"); WebDriver driver = new ChromeDriver(); String url = "http://www.upmandiparishad.in/commodityWiseAll.aspx"; driver.get(url); Thread.sleep(5000); new Select(driver.findElement(By.id("ctl00_ContentPlaceHolder1_ddl_commodity"))).selectByVisibleText("Paddy"); Thread.sleep(5000); driver.findElement(By.id("ctl00_ContentPlaceHolder1_txt_rate")).sendKeys("27/03/2014"); Thread.sleep(5000); driver.findElement(By.id("ctl00_ContentPlaceHolder1_btn_show")).click(); Thread.sleep(5000); driver.close();
}
它可以与FireFox(29.0)和Chrome(34.0)驱动程序正常工作。我使用了以下jar-s:
您应该检查项目的构建路径并添加适当的jar。但是,如果打开它,则可以在selenium-server- standalone-2.XY.0.jar中找到丢失的类文件:selenium-server-standalone-2.41.0.jar \ org \ apache \ http \ entity \ ContentType
另一种方法是使用包含缺少的类文件的httpcore.jar(httpcore-4.3.jar)。